Title text: In the 60s, Marvin Minsky assigned a couple of undergrads to spend the summer programming a computer to use a camera to identify objects in a scene. He figured they'd have the problem solved by the end of the summer. Half a century later, we're still working on it.
Cueball appears to be asking Ponytail to write an app that determines if a given picture is (1) taken in a national park, and (2) a picture of a bird. The first question is generally harder for a human to answer, but easy for an app that has access to location information and a geographic information system (GIS). The second one is easy for a human but much harder for a computer. This illustrates Moravec's paradox from the 1980s in a modern context. By the 1950s computers were useful for tasks like trajectory optimization, generating novel mathematical proofs and the game of checkers, so such high-level computation and reasoning tasks that were hard for humans turned out to be relatively easy for them. On the other hand it turns out to be hard to "give them the skills of a one-year-old when it comes to perception", as Moravec wrote.
In order to determine whether the user is in a national park, Ponytail plans to determine the user's location using the mobile device. This location will then be cross checked with a geographic information system (GIS) which will be able to determine whether the coordinates lie within a national park boundary.
Determining whether an image is of a given kind of natural object is far more difficult. This task falls into the area of computer vision. One of the goals in computer vision is to detect and classify objects within an image. This is a very challenging task for a number of reasons.
- Firstly, humans use size, edge-assignment, movement, and stereoscopic vision when looking at a scene (not a picture of a thing, but of the thing itself) to discern individual objects and then categorize them as foreground or background. A photograph, however, is a static, monoscopic image that can only provide size and edge-assignment clues. Humans are only able to discern objects from background in photographs by comparing the photo against all of the things they've seen and everything they've learned about those things over the course of their life and identifying matching patterns. Presumably, today's computers do not have nearly the processing power or wealth of data available as the human mind.
- Secondly, the quality of the photograph will have an impact on a computer's ability to match patterns. For example, the object in the photograph might be partially visible or occluded. In the case of a living bird, additional complications arise from the variations among individual birds of the same species and differences in pose (flying, perching in a tree, etc.). Differentiating between visually similar objects can result in false positives. For example, is it a photo of a bird in flight or a plane (or superman!)? Ponytail's estimate of 5 years may be overly optimistic (see 678: Researcher Translation).
Today's state-of-the-art algorithms for solving this kind of task mostly use local features (e.g. SIFT or SURF in combination with a support vector machine or convolutional neural network).
The subtitle refers to "CS", which is a common acronym for "Computer Science", of which artificial intelligence and computer vision are sub-disciplines.
The title text mentions The Summer Vision Project and Marvin Minsky of MIT. In the summer of 1966, he asked his undergraduate student Gerald Jay Sussman to "spend the summer linking a camera to a computer and getting the computer to describe what it saw" (). Seymour Papert drafted the plan, and it seems that Sussman was joined by Bill Gosper, Richard Greenblatt, Leslie Lamport, Adolfo Guzman, Michael Speciner, John White, Benjamin, and Henneman. The project schedule allocated one summer for the completion of this task. The required time was obviously significantly underestimated, since dozens of research groups around the world are still working on this topic today.
A month this comic came out, Flickr released a prototype online tool to do just what this comic describes, using its automated-tagging software to answer the bird question.
- [Ponytail sitting at a computer with Cueball standing behind her]
- Cueball: When a user takes a photo, the app should check whether they're in a national park...
- Ponytail: Sure, easy GIS lookup. Gimme a few hours.
- Cueball: ...and check whether the photo is of a bird.
- Ponytail: I'll need a research team and five years.
- In CS, it can be hard to explain the difference between the easy and the virtually impossible.
add a comment! ⋅ add a topic (use sparingly)! ⋅ refresh comments!
the source of title text maybe is Szeliski, Computer Vision: Algorithms and Applications (2010), p. 10. --valepert (talk) 06:59, 24 September 2014 (UTC)
Google’s Artificial Brain Learns to Find Cat Videos might be useful as a description of the problem 188.8.131.52 08:34, 24 September 2014 (UTC)
- Sorry for editing your comment but external links have different syntax that internal links so it wasn't working. -- Hkmaly (talk) 11:21, 24 September 2014 (UTC)
Nice Superman joke there, Pudder! --184.108.40.206 10:26, 24 September 2014 (UTC)
- It had been removed in an edit, so I shoehorned in back in :P --Pudder (talk) 12:25, 24 September 2014 (UTC)
Isn't there an xkcd where the estimate of 5 years of work is equivalent to "might take forever?" Rtanenbaum (talk) 13:16, 24 September 2014 (UTC)
- I'm pretty sure you're refering to 678. 220.127.116.11 15:00, 25 September 2014 (UTC)
The link in the description is to a document by Seymour Papert and the book on the project is also by Papert. Is there any contemporary evidence that it was actually Minsky who assigned the project? I think he just got interested in it later. 14:17, 24 September 2014 (UTC)
678: Researcher Translation is probably what you're thinking of, Rtanenbaum. Ndgeek (talk) 17:44, 24 September 2014 (UTC)
Is it possible that Randall's selection of bird rather than another naturally occurring object is a subtle reference to the Birdsnap app (http://engineering.columbia.edu/it-crow-or-raven-new-birdsnap-app-will-tell-you-0) which has solved some of the aspects of this problem? 18.104.22.168 22:02, 27 September 2014 (UTC)
Hopefully I can add that this also seems to make reference to the U.S. Forest Service intention to make everyone have a permit to take pics, etc., in national parks. https://www.yahoo.com/travel/dont-take-that-picture-the-u-s-forest-service-might-98484656432.html 22.214.171.124 (talk) (please sign your comments with ~~~~)
Post the picture to an online forum, say it's a bird, if it's not everyone will correct you as per http://xkcd.com/386/, so scrape forum and if there's a lot of attention it's not a bird, if there isn't much attention it probably is a bird. 126.96.36.199 23:06, 3 October 2014 (UTC)
A dev team at Flickr took this comic as a challenge, and set up a PoC at http://parkorbird.flickr.com/ (that seems to work fairly well). --188.8.131.52 20:08, 20 October 2014 (UTC)
- I was duly impressed. It doesn't recognize big bird very well, though. ;) Suspender guy (talk) 20:26, 17 February 2016 (UTC)
A 'picture of a bird' from a CS perspective is a reverse engineering problem. The picture is a 2 dimensional rendering of a 3-dimensional world and a 'bird' is a 3-dimensional object. It takes years for the mind of a newborn human to be able to recognize a majority of objects based on their 'first look' at a stereoscopic (two-eyes) image presented by their visual cortex. The software equivalency of this would be to create a 3 dimensional representation of objects and create a linear-algebra algorithm that can define the statistical probability that any given shape is within a certain degree of exclusion a matrix representation of the target shape (area) of the 3 dimensional object (bird) based on distance (using spacial reconstruction). It's not impossible, it's just really really hard. - nerd answer 184.108.40.206 (talk) (please sign your comments with ~~~~)
- To be honest I don't think it is impossible to replicate any function of human intelligence and mental capacity on a computer system. It just requires sufficient processing ability, appropriate hardware, and of course, an understanding of how humans do it in the first place. -Pennpenn 220.127.116.11 03:29, 17 September 2015 (UTC)
Or just give Google a little less than two years, and they'll make Google Cloud Vision API for you Gpk (talk) 20:39, 13 June 2016 (UTC)
I read somewhere that when you ask CS/IT specialist for a probable ETA for solving an interesting problem, you need to multiply the given time to the ETA by 4 and take the next larger unit (a minute becomes 4 hours, an hour becomes 4 days etc.). Can't find the source of that though. 18.104.22.168 15:47, 12 September 2016 (UTC)