1 Introduction 1 How is the brain able to learn about its environment from the sensory input it receives? 2 This fundamental question has been driving neuroscientific research since its beginnings 3 and has been addressed in various domains [19] [32] [42] [18] [28]. Experimental 4 evidence from neuroimaging studies revealed that parts of the brain encode specific 5 sensory stimuli, showing enhanced activity upon presentation of the respective 6 stimulus [21] [34] [13]. Furthermore, similar activation patterns have been observed 7 when subjects had to perform mental tasks involving such stimuli, without the stimuli 8 1/18 actually being presented [39] [14]. While this suggests that the brain does form 9 representations of its environment to perform tasks such as recognition, it does not 10 explain how these representations are formed or what information they are based 11 on [37]. Some information on their nature arose from studies showing them to be 12 hierarchically organized in different sensory domains, ranging from simple sensory to 13 more complex and abstract representations [43] [34] [42] [18]. Given such a hierarchical 14 structure, it seems likely that the representations at a given level are based on the 15 activity of neural populations at the preceding level. As it is a known feature of the 16 cortex to be locally recurrently organized [41] [17], the question arises how such 17 representations can be formed based on the activity of recurrently connected neural 18 populations. This question is of particular interest for the case of dynamic sensory 19 input, i.e. signals changing over time. To encode and decode dynamic sensory signals it 20 is necessary to keep some memory of past input, since the input at a given point in time 21 might not be uniquely assignable to a certain signal. While this condition can be 22 satisfied by the membrane potential at the level of single neurons, it can be satisfied by 23 recurrent connections at the level of neural populations. We were particularly interested 24 in the question, whether it is possible to encode and decode multiple dynamic sensory 25 signals with a strongly simplified model of a recurrently connected neural circuit. More 26 specifically, we wanted to investigate whether dynamic signals, as they occur naturally, 27 can be learned by a recurrent neural network (RNN) with very simple neuron models. 28The crucial task the model has to perform for this purpose, is to extract information 29 about the input to the RNN from its continuously changing activation patterns. This 30 involves observing the state dynamics of the network and detecting patterns in those 31 dynamics that are specific to a certain input signal. Once learned, we wanted to use 32 these representations for recognition of the respective sensory signals. A possible 33 mechanism for the recognition task is proposed by predictive coding theory [12]. It 34 states that internal representations are compared to current sensory input, resulting in a 35 difference signal [3]. This signal in turn i...

No abstract

In the last decades, a large diversity of automatic, semi-automatic and manual approaches for video segmentation and knowledge extraction from video-data has been proposed. Due to the high complexity in both the spatial and temporal domain, it continues to be a challenging research area. In order to develop, train, and evaluate new algorithms, ground truth of video-data is crucial. Pixel-wise annotation of ground truth is usually time-consuming, does not contain semantic relations between objects and uses only simple geometric primitives. We provide a brief review of related tools for video annotation, and introduce our novel interactive and semi-automatic segmentation tool iSeg. Extending an earlier implementation, we improved iSeg with a semantic time line, multithreading and the use of ORB features. A performance evaluation of iSeg on four data sets is presented. Finally, we discuss possible opportunities and applications of semantic polygon-shaped video annotation, such as 3D reconstruction and video inpainting.

No abstract

No abstract