Image software identifies objects

Department of Computer Science Associate Professor Randal Nelson played a major role in developing an innovative type of image recognition software. He is currently working to upgrade and apply it to the Smart Medical Home at the Center for Future Health. In collaboration with other engineers, computer scientists and health specialists at the Center for Future Health, Nelson has integrated an artificial intelligence program he created called “Memory Assistant Technology” into a controlled environment in an attempt to improve daily life. “As people get older and start to develop the disorders of old age, such as Alzheimer’s, they often have trouble remembering simple things, like where you had put your coffee cup,” Nelson said. “This can be extremely frustrating and this can increase levels of depression, so the idea is to use new technology in artificial intelligence, particularly in object recognition and scene analysis, to help people keep track of objects that are easily misplaced.”Nelson demonstrated the program by placing a random object – in this trial, a bright pink plastic bear – on a table approximately four feet from the small black camera. On the computer screen, which showed the monitored view, the newly placed bear glowed green to highlight the new object in the scene.”You have to train a computer to know what an object looks like and show it from every point of view. It takes several pictures of the bear sitting on the desk in different positions and it stores the image in its memory,” he said. “Nobody knows how recognition in an animal works so I can’t compare, but the program has to be taught like a person, by showing. There is no mathematical model of the bear and it would probably be harder to figure that out rather than just showing it to a computer.”Doctorate candidate and graduate student under Nelson’s direction Brandon Sanders focuses on “spatial intelligence,” which is one’s ability to perceive and reason about objects and actions involving them. “When we make plans and converse with our friends we think in terms of objects. Object names, object characteristics, relationships between objects – we can scarcely think a thought or speak a sentence that is not fundamentally about objects,” he said. “When we hook a camera up to a computer the computer doesn’t see objects, it sees a big batch of numbers. Object Discovery is about teaching the computer to determine which of those numbers belong together. We want to help the computer ‘discover’ the objects in the numbers.”Although basic artificial intelligence systems already exist, such as the cruise control option in modern cars or thermostat technology in houses, both of which have goals and are in some basic way aware of their surroundings, Nelson said that he sees this program as a possible upgrade in computer consciousness. “This kind of idea of consciousness is pretty rudimentary, self-consciousness, however, is very different and refers to an awareness of one’s existence rather than simply responding. But if you sit in front of it, it will take a picture, follow you around, and I swear you will get the feeling that it is conscious, and aware of you, but still it is not aware of you,” he said. “Computers live in a world of signals coming down from wire, and they don’t process stuff like sounds, like our wires do, which is probably why a lot of people don’t understand computers.”According to Nelson, the program will recognize and hone in on distinctive parts of an object, such as the curve of a cup, an almost definitive characteristic of coffee mugs and not easily mimicked by other distracting objects.”This whole process of closing in on specific features made me think of Picasso paintings. We are also working on face recognition, so you can see how close it is to something like what Picasso was working on,” he said. “How far can I distort this image before people cannot recognize it? And that’s exactly what we’re doing to test the program, distorting the relative positions of an object. It might take 100 pictures from each angle for the program to recognize the object.”Nelson originally pursued work in physics, but he received his masters and his doctorate in computer science from the University of Maryland.Welzer can be reached at bwelzer@campustimes.org.