MIT-and-Toyota-release-innovative-dataset-to-accelerate-autonomous-driving.png

MIT and Toyota release innovative dataset to accelerate autonomous driving research | MIT News

news .mit.edu/sites/default/files/styles/news_article__cover_image__original/public/images/202006/MIT-AgeLab-DriveSeg-Open-DataSet.png?itok=p_jbXFXI” />

The following was issued as a joint release from the MIT AgeLab and Toyota Collaborative Safety Research Center.

How can we train self-driving vehicles to have a deeper awareness of the world around them? Can computers learn from past experiences to recognize future patterns that can help them safely navigate new and unpredictable situations?

These are some of the questions researchers from the AgeLab at the MIT Center for Transportation and Logistics and the Toyota Collaborative Safety Research Center (CSRC) are trying to answer by sharing an innovative new open dataset called DriveSeg.

Through the release of DriveSeg, MIT and Toyota are working to advance research in autonomous driving systems that, much like human perception, perceive the driving environment as a continuous flow of visual information.

“In sharing this dataset, we hope to encourage researchers, the industry, and other innovators to develop new insight and direction into temporal AI modeling that enables the next generation of assisted driving and automotive safety technologies,” says Bryan Reimer, principal researcher. “Our longstanding working relationship with Toyota CSRC has enabled our research efforts to impact future safety technologies.”

“Predictive power is an important part of human intelligence,” says Rini Sherony, Toyota CSRC’s senior principal engineer. “Whenever we drive, we are always tracking the movements of the environment around us to identify potential risks and make safer decisions. By sharing this dataset, we hope to accelerate research into autonomous driving systems and advanced safety features that are more attuned to the complexity of the environment around them.”

To date, self-driving data made available to the research community have primarily consisted of troves of static, single images that can be used to identify and track common objects found in and around the road, such as bicycles, pedestrians, or traffic lights, through the use of “bounding boxes.” By contrast, DriveSeg contains more precise, pixel-level representations of many of these same common road objects, but through the lens of a continuous video driving scene. This type of full-scene segmentation can be particularly helpful for identifying more amorphous objects — such as road construction and vegetation — that do not always have such defined and uniform shapes.

According to Sherony, video -based driving scene perception provides a flow of data that more closely resembles dynamic, real-world driving situations. It also allows researchers to explore data patterns as they play out over time, which could lead to advances in machine learning, scene understanding, and behavioral prediction.

DriveSeg is available for free and can be used by researchers and the academic community for non-commercial purposes at the links below. The data is comprised of two parts. DriveSeg (manual) is 2 minutes and 47 seconds of high-resolution video captured during a daytime trip around the busy streets of Cambridge, Massachusetts. The video ’s 5,000 frames are densely annotated manually with per-pixel human labels of 12 classes of road objects.

DriveSeg (Semi-auto) is 20,100 video frames (67 10-second video clips) drawn from MIT Advanced Vehicle Technologies (AVT) Consortium data. DriveSeg (Semi-auto) is labeled with the same pixel-wise semantic annotation as DriveSeg (manual), except annotations were completed through a novel semiautomatic annotation approach developed by MIT. This approach leverages both manual and computational efforts to coarsely annotate data more efficiently at a lower cost than manual annotation. This dataset was created to assess the feasibility of annotating a wide range of real-world driving scenarios and assess the potential of training vehicle perception systems on pixel labels created through AI-based labeling systems.

To learn more about the technical specifications and permitted use-cases for the data, visit the DriveSeg dataset page.

Covid Abruzzo Basilicata Calabria Campania Emilia Romagna Friuli Venezia Giulia Lazio Liguria Lombardia Marche Molise Piemonte Puglia Sardegna Sicilia Toscana Trentino Alto Adige Umbria Valle d’Aosta Veneto Italia Agrigento Alessandria Ancona Aosta Arezzo Ascoli Piceno Asti Avellino Bari Barletta-Andria-Trani Belluno Benevento Bergamo Biella Bologna Bolzano Brescia Brindisi Cagliari Caltanissetta Campobasso Carbonia-Iglesias Caserta Catania Catanzaro Chieti Como Cosenza Cremona Crotone Cuneo Enna Fermo Ferrara Firenze Foggia Forlì-Cesena Frosinone Genova Gorizia Grosseto Imperia Isernia La Spezia L’Aquila Latina Lecce Lecco Livorno Lodi Lucca Macerata Mantova Massa-Carrara Matera Messina Milano Modena Monza e della Brianza Napoli Novara Nuoro Olbia-Tempio Oristano Padova Palermo Parma Pavia Perugia Pesaro e Urbino Pescara Piacenza Pisa Pistoia Pordenone Potenza Prato Ragusa Ravenna Reggio Calabria Reggio Emilia Rieti Rimini Roma Rovigo Salerno Medio Campidano Sassari Savona Siena Siracusa Sondrio Taranto Teramo Terni Torino Ogliastra Trapani Trento Treviso Trieste Udine Varese Venezia Verbano-Cusio-Ossola Vercelli Verona Vibo Valentia Vicenza Viterbo

Research-into-AI-Neuroscience-Psychology-Aims-to-Make-AI-Less-Artificial.jpg

Research into AI, Neuroscience, Psychology Aims to Make AI Less Artificial 

AI and neuroscience are developing together, making each stronger. (Source: Getty Images) 

By AI Trends Staff  

Research at the intersection of AI, psychology, and neuroscience is attracting interest and investment. The study of the nervous system is called by some the “ultimate challenge” of the biological sciences.    

Irina Rish, Associate Professor, Computer Science and Operations Research department, Université de Montréal, and a core member of Mila – the Quebec AI Institute

The trend is exemplified in the experience of Irina Rish, now an Associate Professor  in the Computer Science and Operations Research department at the Université de Montréal (UdeM),and a core member of Mila – the Quebec AI Institute.   

Rish was 14 years old and going to high school in the central Asian city of Samarkand, Uzbekistan, when she first came across the notion of artificial intelligence. “I saw a book, translated from English into Russian, the cover was black with yellow letters, and the title was ‘Can Machines Think?’” Rish recalled in a recent article in Mirage.   

Rish was intrigued. “The book was about AI, and I said to myself: ‘Gosh, that’s exactly what I was wondering: what algorithms can we design to solve difficult problems, and how can we boost our own ‘natural intelligence,’” she recalled. 

That curiosity set her on a path to her life’s work. She graduated from universities in Moscow and California, then embarked on what became a 20-year career at IBM, including 10 years as a research scientist at the Watson Research Center. Last October, she moved to Canada to become an associate professor at Université de Montréal and a core faculty member at its affiliated AI institute, Mila. 

This summer she was awarded a Canada Excellence Research Chair (CERC), which came with a $34 million grant over several years from the federal government and other sources, including industry players Samsung, IBM, Microsoft, and Element AI.  

“It’s a wonderful opportunity for me and my team at Mila,” Rish said. “Over the coming years, this chair will allow us to explore the frontiers of AI research at the intersection of machine learning and neuroscience, and advance the field toward more autonomous, human-level AI by developing novel models and methods for broad and robust AI systems, as opposed to today’s narrow and brittle ones,” she said. 

Rish holds 64 patents, has published over 90 research papers, written several book chapters, edited three books and published a monograph on sparse modelling, an area of statistical machine learning particularly important for scientific data analysis such as computational biology and neuroimaging. 

So far at Mila, among her projects has been working with scientific director Yoshua Bengio to help develop Covi, a contact-tracing app for Covid-19.  

Her goal is “to develop continual, lifelong learning AI capabilities, similar to those of humans, as well as approaches to making AI more robust to changes in its environment and tasks it has to solve, and capable of better understanding and generalization, akin to human capabilities,” she stated.  

She sees the work as “the intersection of artificial intelligence, neuroscience, and psychology, using computers to analyze brain data and find interesting patterns there related to human behavior, to mental states and their changes, and using what you learn to better understand how the brain works and to make computers work better and AI less artificial.” 

Using AI to Decode How the Brain Sends Signals to Limbs  

Chethan Pandarinath, biomedical engineer at Emory University and the Georgia Institute of Technology

Researcher Chethan Pandarinath, a biomedical engineer at Emory University and the Georgia Institute of Technology, both in Atlanta, is working on enabling people with paralyzed limbs to reach out and grasp with a robotic arm as they would their own. He is collecting recordings of brain activity in people with paralysis, in the hopes of identifying patterns of electrical activity in neurons that correspond to moving an arm in a particular way, so that the instruction can be fed to an artificial limb. That is akin to reading minds.  

“It turns out, that’s a really challenging problem,” Pandarinath stated in a recent account in Nature. “These signals from the brain—they’re really complicated.” He decided to feed his brain activity recordings into an artificial neural network, a software architecture inspired by the brain, to try to get it to reproduce the data.   

Patterns the researchers call latent factors were found to control the overall behavior of the recorded activity. The effort revealed the brain’s temporal dynamics, the way a pattern of neural activity changes from one moment to the next. This allowed a more fine-grained set of instructions to be produced for arm movements than previous methods. “Now, we can very precisely say, on an almost millisecond-by-millisecond basis, right now the animal is trying to move at this precise angle,” Pandarinath stated. “That’s exactly what we need to know to control a robotic arm.” 

In this way, AI is helping brain science and brain science is giving more insight to AI researchers. “The technology is coming full circle and being applied back to understand the brain,” he stated.  

An artificial neural network is only a rough analogy of how the brain works, stated David Sussillo, a computational neuroscientist with the Google Brain Team in San Francisco, who collaborated with Pandarinath on his work on latent factors. For instance, it models synapses as numbers in a matrix, when in reality they are complex pieces of biological machinery that use both chemical and electrical activity to send or terminate signals, and that interact with their neighbors in dynamic patterns. “You couldn’t get further from the truth of what a synapse actually is than a single number in a matrix,” Sussillo stated. 

Still, artificial neural networks have proved useful for studying the brain. If such a system can produce a pattern of neural activity that resembles the pattern that is recorded from the brain, scientists can examine how the system generates its output and then make inferences about how the brain does the same thing. This approach can be applied to any cognitive task of interest to neuroscientists, including processing an image. “If you can train a neural network to do it,” stated Sussillo, “then perhaps you can understand how that network functions, and then use that to understand the biological data.”  

Comparing How Machine Learning Works to How the Brain Works 

Gabriel A. Silva, professor of Bioengineering and Neurosciences, University of California, San Diego

A similar conclusion was reached by Gabriel A. Silva, a professor of Bioengineering and Neurosciences at the University of California, San Diego, whose work includes how study of the brain can have practical benefits for new AI systems.  

“I and other researchers in the field, including a number of its leaders, have a growing sense that finding out more about how the brain processes information could help programmers translate the concepts of thinking from the wet and squishy world of biology into all-new forms of machine learning in the digital world,” Silva stated in an article in Neuroscience News.   

How machine learning works and how the brain works are very different. To recognize an image of a cow, a machine learning system needs to be fed many, many images of cows in order to learn. Whereas, “The brain takes in a very small amount of input data—like a photograph of a cow and a drawing of a cow—very quickly. And after only a very small number of examples, even a toddler will grasp the idea of what a cow looks like and be able to identify one in new images, from different angles, and in different colors,” Silva wrote. 

The brain and machine learning systems use fundamentally different algorithms. Because of this, “each excels in ways the other fails miserably,” Silva observes.  

It is challenging to try to distinguish which brain processes might work well as machine learning algorithms. One approach is to focus on ideas that improve machine learning and identify new areas of neuroscience, at the same time.  

“Lessons can go both ways, from brain science to artificial intelligence—and back, with AI research highlighting new questions for biological neuroscientists,” Silva suggests.  

Read the source articles in Mirage, in Nature and in Neuroscience News. 

Covid Abruzzo Basilicata Calabria Campania Emilia Romagna Friuli Venezia Giulia Lazio Liguria Lombardia Marche Molise Piemonte Puglia Sardegna Sicilia Toscana Trentino Alto Adige Umbria Valle d’Aosta Veneto Italia Agrigento Alessandria Ancona Aosta Arezzo Ascoli Piceno Asti Avellino Bari Barletta-Andria-Trani Belluno Benevento Bergamo Biella Bologna Bolzano Brescia Brindisi Cagliari Caltanissetta Campobasso Carbonia-Iglesias Caserta Catania Catanzaro Chieti Como Cosenza Cremona Crotone Cuneo Enna Fermo Ferrara Firenze Foggia Forlì-Cesena Frosinone Genova Gorizia Grosseto Imperia Isernia La Spezia L’Aquila Latina Lecce Lecco Livorno Lodi Lucca Macerata Mantova Massa-Carrara Matera Messina Milano Modena Monza e della Brianza Napoli Novara Nuoro Olbia-Tempio Oristano Padova Palermo Parma Pavia Perugia Pesaro e Urbino Pescara Piacenza Pisa Pistoia Pordenone Potenza Prato Ragusa Ravenna Reggio Calabria Reggio Emilia Rieti Rimini Roma Rovigo Salerno Medio Campidano Sassari Savona Siena Siracusa Sondrio Taranto Teramo Terni Torino Ogliastra Trapani Trento Treviso Trieste Udine Varese Venezia Verbano-Cusio-Ossola Vercelli Verona Vibo Valentia Vicenza Viterbo

MIT-undergraduates-pursue-research-opportunities-through-the-pandemic-MIT.png

MIT undergraduates pursue research opportunities through the pandemic | MIT News

news .mit.edu/sites/default/files/styles/news_article__cover_image__original/public/images/202009/Summer-UROP.png?itok=OyxvrrJc” />

Even in ordinary times, scientific process is stressful, with its demand for open-ended exploration and persistence in the face of failure. But the pandemic has added to the strain. In this new world of physical isolation, there are fewer opportunities for spontaneity and connection, and fewer distractions and events to mark the passage of time. Days pass in a numbing blur of sameness.

Working from home this summer, students participating in MIT’s Undergraduate Research Opportunities Program (UROP) did their best to overcome these challenges. Checking in with their advisors over Zoom and Slack, from as far west as Los Angeles, California and as far east as Skopje, North Macedonia, they completed two dozen projects sponsored by the MIT Quest for Intelligence. Four student projects are highlighted here.

Defending code-processing AI models against adversarial attacks 

Computer vision models have famously been fooled into classifying turtles as rifles, and planes as pigs, simply by making subtle changes to the objects and images the models are asked to interpret. But models that analyze computer code, which are a part of recent efforts to build automated tools to design programs efficiently, are also susceptible to so-called adversarial examples. 

The lab of Una-May O’Reilly, a principal research scientist at MIT, is focused on finding and fixing the weaknesses in code-processing models that can cause them to misbehave. As automated programming methods become more common, researchers are looking for ways to make this class of deep learning model more secure.

“Even small changes like giving a different name to a variable in a computer program can completely change how the model interprets the program,” says Tamara Mitrovska, a third-year student who worked on a UROP project this summer with Shashank Srikant, a graduate student in O’Reilly’s lab.

The lab is investigating two types of models used to summarize bits of a program as part of a broader effort to use machine learning to write new programs. One such model is Google’s seq2seq, originally developed for machine translation. A second is code2seq, which creates abstract representations of programs. Both are vulnerable to attacks due to a simple programming quirk: captions that let humans know what the code is doing, like assigning names to variables, give attackers an opening to exploit the model. By simply changing a variable name in a program or adding a print statement, the program may function normally, yet force the model processing it to give an incorrect answer.

This summer, from her home near Skopje, in North Macedonia, Mitrovska learned how to sift through a database of more than 100,000 programs in Java and Python and modify them algorithmically to try to fool seq2seq and code2seq. “These systems are challenging to implement,” she says. “Finding even the smallest bug can take a significant amount of time. But overall, I’ve been having fun and the project has been a very good learning experience for me.”

One exploit that she uncovered: Both models could be tricked by inserting “print” commands in the programs they process. That exploit, and others discovered by the lab, will be used to update the models to make them more robust.

What everyday adjectives can tell us about human reasoning

Embedded in the simplest of words are assumptions about the world that vary even among closely related languages. Take the word “biggest.” Like other superlatives in English, this adjective has no equivalent in French or Spanish. Speakers simply use the comparative form, “bigger” — plus grand in French or más grande in Spanish — to differentiate among objects of various sizes.

To understand what these words mean and how they are actually used, Helena Aparicio, formerly a postdoc at MIT and now a professor at Cornell University, devised a set of psychology experiments with MIT Associate Professor Roger Levy and Boston University Professor Elizabeth Coppock. Curtis Chen, a second-year student at MIT interested in the four topics that converge in Levy’s lab — computer science, psychology, linguistics, and cognitive science — joined on as a UROP student.

From his home in Hillsborough, New Jersey, Chen orchestrated experiments to identify why English speakers prefer superlatives in some cases and comparatives in others. He found that in scenes with more similarly sized objects, the more likely his human subjects were to prefer the word “biggest” to describe the largest object in the set. When objects appeared to fall within two clearly defined groups, subjects preferred the less-precise “bigger.” Chen also built an AI model to simulate the inferences made by his human subjects and found that it showed a similar preference for the superlative in ambiguous situations.

Designing a successful experiment can take several tries. To ensure consistency among the shapes that subjects were asked to describe, Chen generated them on the computer using HTML Canvas and JavaScript. “This way, the size differentials were exact, and we could simply report the formula used to make them,” he says.

After discovering that some subjects seemed confused by rectangle and line shapes, he replaced them with circles. He also removed the default option on his reporting scale after realizing that some subjects were using it to breeze through the tasks. Finally, he switched to the crowdsourcing platform Prolific after a number of participants on Amazon’s Mechanical Turk failed at tasks designed to ensure they were taking the experiments seriously.

“It was discouraging, but Curtis went through the process of exploring the data and figuring out what was going wrong,” says his mentor, Aparicio. 

In the end, he wound up with strong results and promising ideas for follow-up experiments this fall. “There’s still a lot to be done,” he says. “I had a lot of fun cooking up and tweaking the model, designing the experiment, and learning about this deceptively simple puzzle.”

Levy says he looks forward to the results. “Ultimately, this line of inquiry helps us understand how different vocabularies and grammatical resources of English and thousands of other languages support flexible communication by their native speakers,” he says.

Reconstructing real-world scenes from sensor data

AI systems that have become expert at sizing up scenes in photos and video may soon be able to do the same for real-world scenes. It’s a process that involves stitching together snapshots of a scene from varying viewpoints into a coherent picture. The brain performs these calculations effortlessly as we move through the world, but computers require sophisticated algorithms and extensive training. 

MIT Associate Professor Justin Solomon focuses on developing methods to help computers understand 3D environments. He and his lab look for new ways to take point cloud data gathered by sensors — essentially, reflections of infrared light bounced off the surfaces of objects — to create a holistic representation of a real-world scene. Three-dimensional scene analysis has many applications in computer graphics, but the one that drove second-year student Kevin Shao to join Solomon’s lab was its potential as a navigation tool for self-driving cars.

“Working on autonomous cars has been a childhood dream for me,” says Shao.

In the first phase of his UROP project, Shao downloaded the most important papers on 3D scene reconstruction and tried to reproduce their results. This improved his knowledge of PyTorch, the Python library that provides tools for training, testing, and evaluating models. It also gave him a deep understanding of the literature. In the second phase of the project, Shao worked with his mentor, PhD student Yue Wang, to improve on existing methods.

“Kevin implemented most of the ideas, and explained in detail why they would or wouldn’t work,” says Wang. “He didn’t give up on an idea until we had a comprehensive analysis of the problem.”

One idea they explored was the use of computer-drawn scenes to train a multi-view registration model. So far, the method works in simulation, but not on real-world scenes. Shao is now trying to incorporate real-world data to bridge the gap, and will continue the work this fall.

Wang is excited to see the results. “It sometimes takes PhD students a year to have a reasonable result,” he says. “Although we are still in the exploration phase, I think Kevin has made a successful transition from a smart student to a well-qualified researcher.”

When do infants become attuned to speech and music?

The ability to perceive speech and music has been traced to specialized parts of the brain, with infants as young as four months old showing sensitivity to speech-like sounds. MIT Professor Nancy Kanwisher and her lab are investigating how this special ear for speech and music arises in the infant brain.

Somaia Saba, a second-year student at MIT, was introduced to Kanwisher’s research last year in an intro to neuroscience class and immediately wanted to learn more. “The more I read up about cortical development, the more I realized how little we know about the development of the visual and auditory pathways,” she says. “I became very excited and met with [PhD student] Heather Kosakowski, who explained the details of her projects.”

Signing on for a project, Saba plunged into the “deep end” of cortical development research. Initially overwhelmed, she says she gained confidence through regular Zoom meetings with Kosakowski, who helped her to navigate MATLAB and other software for analyzing brain-imaging data. “Heather really helped motivate me to learn these programs quickly, which has also primed me to learn more easily in the future,” she says.

Before the pandemic shut down campus, Kanwisher’s lab collected functional magnetic resonance imaging (fMRI) data from two- to eight-week-old sleeping infants exposed to different sounds. This summer, from her home on Long Island, New York, Saba helped to analyze the data. She is now learning how to process fMRI data for awake infants, looking toward the study’s next phase. “This is a crucial and very challenging task that’s harder than processing child and adult fMRI data,” says Kosakowski. “Discovering how these specialized regions emerge in infants may be the key to unlocking mysteries about the origin of the mind.”

MIT Quest for Intelligence summer UROP projects were funded, in part, by the MIT-IBM Watson AI Lab and by Eric Schmidt, technical advisor to Alphabet Inc., and his wife, Wendy.

Covid Abruzzo Basilicata Calabria Campania Emilia Romagna Friuli Venezia Giulia Lazio Liguria Lombardia Marche Molise Piemonte Puglia Sardegna Sicilia Toscana Trentino Alto Adige Umbria Valle d’Aosta Veneto Italia Agrigento Alessandria Ancona Aosta Arezzo Ascoli Piceno Asti Avellino Bari Barletta-Andria-Trani Belluno Benevento Bergamo Biella Bologna Bolzano Brescia Brindisi Cagliari Caltanissetta Campobasso Carbonia-Iglesias Caserta Catania Catanzaro Chieti Como Cosenza Cremona Crotone Cuneo Enna Fermo Ferrara Firenze Foggia Forlì-Cesena Frosinone Genova Gorizia Grosseto Imperia Isernia La Spezia L’Aquila Latina Lecce Lecco Livorno Lodi Lucca Macerata Mantova Massa-Carrara Matera Messina Milano Modena Monza e della Brianza Napoli Novara Nuoro Olbia-Tempio Oristano Padova Palermo Parma Pavia Perugia Pesaro e Urbino Pescara Piacenza Pisa Pistoia Pordenone Potenza Prato Ragusa Ravenna Reggio Calabria Reggio Emilia Rieti Rimini Roma Rovigo Salerno Medio Campidano Sassari Savona Siena Siracusa Sondrio Taranto Teramo Terni Torino Ogliastra Trapani Trento Treviso Trieste Udine Varese Venezia Verbano-Cusio-Ossola Vercelli Verona Vibo Valentia Vicenza Viterbo

Recent Posts

Archives