Today, and possibly for a long time to come, the full driving task is too complex an activity to be fully formalized as a sensing-acting robotics system that can be explicitly solved through model-based and learning-based approaches in order to achieve full unconstrained vehicle autonomy. Localization, mapping, scene perception, vehicle control, trajectory optimization, and higher-level planning decisions associated with autonomous vehicle development remain full of open challenges. This is especially true for unconstrained, real-world operation where the margin of allowable error is extremely small and the number of edge-cases is extremely large. Until these problems are solved, human beings will remain an integral part of the driving task, monitoring the AI system as it performs anywhere from just over 0% to just under 100% of the driving. The governing objectives of the MIT Advanced Vehicle Technology (MIT-AVT) study are to (1) undertake large-scale real-world driving data collection that includes high-definition video to fuel the development of deep learning based internal and external perception systems, (2) gain a holistic understanding of how human beings interact with vehicle automation technology by integrating video data with vehicle state data, driver characteristics, mental models, and self-reported experiences with technology, and (3) identify how technology and other factors related to automation adoption and use can be improved in ways that save lives. In pursuing these objectives, we have instrumented 23 Tesla Model S and Model X vehicles, 2 Volvo S90 vehicles, 2 Range Rover Evoque, and 2 Cadillac CT6 vehicles for both long-term (over a year per driver) and medium term (one month per driver) naturalistic driving data collection. Furthermore, we are continually developing new methods for analysis of the massive-scale dataset collected from the instrumented vehicle fleet. The recorded data streams include IMU, GPS, CAN messages, and high-definition video streams of the driver face, the driver cabin, the forward roadway, and the instrument cluster (on select vehicles). The study is on-going and growing. To date, we have 122 participants, 15,610 days of participation, 511,638 miles, and 7.1 billion video frames. This paper presents the design of the study, the data collection hardware, the processing of the data, and the computer vision algorithms currently being used to extract actionable knowledge from the data. 01 231 4523 67 89 8 %& 'ÿ )*+ ,-,.,*/ÿ 0123 45 1 '142-,5 ,67ÿ 8+ *97 :; <=>ÿ @AB; CDÿ ; AE =F; GH ÿIJ KFL; M NM OFB; ÿ =F>DH ÿPQR SPT ULM VGLDH ÿPWW XGCM NY GDH ÿWZ [M Y GDÿ =LM VGBH ÿQPPR SI\ XM =GAÿ ] LF@GDH ÿJPPÿa b b a cd :; <=>ÿ =F; Fÿ NAY Y GN; M ABÿ M Dÿ ABeAM Bef ÿ :; F; M D; M NDÿ

We introduce a recurrent neural network architecture for automated road surface wetness detection from audio of tiresurface interaction. The robustness of our approach is evaluated on 785,826 bins of audio that span an extensive range of vehicle speeds, noises from the environment, road surface types, and pavement conditions including international roughness index (IRI) values from 25 in/mi to 1400 in/mi. The training and evaluation of the model are performed on different roads to minimize the impact of environmental and other external factors on the accuracy of the classification. We achieve an unweighted average recall (UAR) of 93.2 % across all vehicle speeds including 0 mph. The classifier still works at 0 mph because the discriminating signal is present in the sound of other vehicles driving by.

We address the puzzle of “unity in diversity” in human languages by advocating the (minimal) common denominator for the diverse expressions of transitivity across human languages, consistent with the view that early in language evolution there was a modest beginning for syntax and that this beginning provided the foundation for the further elaboration of syntactic complexity. This study reports the results of a functional MRI experiment investigating differential patterns of brain activation during processing of sentences with minimal versus fuller syntactic structures. These structural layers have been postulated to represent different stages in the evolution of syntax, potentially engaging different brain networks. We focused on the Serbian “middles,” analyzed as lacking the transitivity (vP) layer, contrasted with matched transitives, containing the transitivity layer. Our main hypothesis was that transitives will produce more activation in the syntactic (Broca's–Basal Ganglia) brain network, in comparison to more rudimentary middles. The participants (n = 14) were healthy adults (Mean age = 33.36; SD = 12.23), native speakers of Serbo-Croatian. The task consisted of reading a series of sentences (middles and transitives; n = 64) presented in blocks of 8, while being engaged in a detection of repetition task. We found that the processing of transitives, compared to middles, was associated with an increase in activation in the basal ganglia bilaterally. Although we did not find an effect in Broca's area, transitives, compared to middles, evoked greater activation in the precentral gyrus (BA 6), proposed to be part of the “Broca's complex.” Our results add to the previous findings that Broca's area is not the sole center for syntactic processing, but rather is part of a larger circuit that involves subcortical structures. We discuss our results in the context of the recent findings concerning the gene-brain-language pathway involving mutations in FOXP2 that likely contributed to the enhancement of the frontal-striatal brain network, facilitating human capacity for complex syntax.

The present fMRI study tested predictions of the evolution-of-syntax framework which analyzes certain structures as remnants (“fossils”) of a non-hierarchical (non-recursive) proto-syntactic stage in the evolution of language (Progovac, 2015, 2016). We hypothesized that processing of these structures, in comparison to more modern hierarchical structures, will show less activation in the brain regions that are part of the syntactic network, including Broca’s area (BA 44 and 45) and the basal ganglia, i.e., the network bolstered in the line of descent of humans through genetic mutations that contributed to present-day dense neuronal connectivity among these regions. Fourteen healthy native English-speaking adults viewed written stimuli consisting of: (1) full sentences (FullS; e.g., The case is closed); (2) Small Clauses (SC; e.g., Case closed); (3) Complex hierarchical compounds (e.g., joy-killer); and (4) Simple flat compounds (e.g., kill-joy). SC (compared to FullS) resulted in reduced activation in the left BA 44 and right basal ganglia. Simple (relative to complex) compounds resulted in increased activation in the inferior temporal gyrus and the fusiform gyrus (BA 37/19), areas implicated in visual and semantic processing. We discuss our findings in the context of current theories regarding the co-evolution of language and the brain.

We propose a method for automated synchronization of vehicle sensors useful for the study of multimodal driver behavior and for the design of advanced driver assistance systems. Multi-sensor decision fusion relies on synchronized data streams in (1) the offline supervised learning context and (2) the online prediction context. In practice, such data streams are often out of sync due to the absence of a real-time clock, use of multiple recording devices, or improper thread scheduling and data buffer management. Cross-correlation of accelerometer, telemetry, audio, and dense optical flow from three video sensors is used to achieve an average synchronization error of 13 milliseconds. The insight underlying the effectiveness of the proposed approach is that the described sensors capture overlapping aspects of vehicle vibrations and vehicle steering allowing the cross-correlation function to serve as a way to compute the delay shift in each sensor. Furthermore, we show the decrease in synchronization error as a function of the duration of the data stream.