After the very successful retreat last year, we decided we wanted to make this a tradition and organize it again. This year it was held in Ofterschwang, Allgäu from May 31 to June 3. During these four days we had 3 guest lectures who gave us about 2 hour lectures each.
Invited Talks
In our Retreat we had the honour to have the presence of Professor Jakob Macke from TU Munich, Professor Benjamin Grewe from the Institute of Neuroinformatics at ETH Zurich and Professor Thomas Euler from the University of Tübingen who talked to us about their research on different aspects of neuroscience.
Professor Macke and his PhD student Artur Speiser talked about their work developing statistical models to model neural activity. Macke’s research intends to understand how neurons and population of neurons process sensory cues from the external world and performs computations on it in order to perform control, behaviour and cognitive tasks.
In order to do that they develop statistical models and machine learning algorithms working with recorded data from live brains as well as simulated neural activity.
Professor Grewe comes from the University of Zurich, more specifically from the Institute of Neuroinformatics. His talk was about the ups and downs of Hebbian plasticity in associative fear learning. During the talk he illustrated the principles behind Calcium imaging, his expertise. Understanding the basis of information processing and memory formation in neuronal networks is his primary research focus.
Our third guest speaker was Professor Thomas Euler. He is the head of The Institute for Opthalmic Research in Universiätsklinikum Tübingen. His research focuses on understanding the function and organization of the retinal microcircuit and the underlying computational principles, as well as the changes of such microcircuits during development and degeneration.
Student Talks
A reasonably big portion of the retreat was devoted to student talks. Overall, we had seven students talks which covered a variety of topics.
Data Visualization with Python by Aleksandra Teska
In the very first student talk, Aleksandra Teska talked about Data Visualization with Python showing off her colorful figures from her research internship project. Her presentation, not only introduced JupyterLab and its useful extension for presentation, but also showed us the importance of data and data visualization in biasing our conclusions.
She finished her presentation by showing us some of her results from her research project, where she took advantage of visualization tools in python to have beautiful informative figures. Her research project was on analysing rats motion while being presented by rotating black and white bars.
A short excursion into psychology: Attention by Annika Thierfelder
Annika, our dearest representative, decided to gain more “attention” by simply talking about it. She illustrated different classic theories of attention, starting with Broadbent’s Early Selection filter theory proposed in 1958, in which attention was viewed as a filtering process - a stream of messages gets filtered based on physical characteristics (e.g., tone, pitch, etc.) and once passed through the filter we attend to the message and the meaning of the message is processed by the detector unit; once the meaning is understood, the message is passed to memory. It is called early selection, because filtering happens relatively early (before the meaning is analyzed) in the process. As we continued with the talk we realized this “early selection” is not what happens in the case of cocktail party - we do not interpret meaning after filtering (and attending to) the information, rather our attention is dragged towards information that is somehow meaningful to us; so filtering could potentially happen after the information was analyzed. Annika continued by talking about another classic theory proposed by Treisman in 1964 which was aiming to solve the shortcomings of Broadbent’s theory, it was called Attenuation theory. In this theory, the filter in Broadbent’s theory was replaced by the Attenuator unit. The attenuator unit analyses information on a hierarchical basis. Once this analysis is done both attended to unattended messages are passed to higher level processing with different weights depending on the amount of attention assigned to specific message(s). This talk was so wonderful that I could continue writing about it several more paragraphs. But if you wanna hear more about this, you know who to talk to. Annika continued by talking more about models of attention (feature integration theory, Capacity theory, etc.). Finally, She tried to finish the talk by relating attention to consciousness! You don’t want to know what happened then, it suffices to say that we had to stop the discussion because we got very short on time.
Behaviour by Werner Seitz
In this talk our colleague Werner Seitz shared with us some experiments about behaviour in animals and how our notion of reality can be strongly influenced by the survival mechanisms that our brain developed.
An example of this is the attention of frog’s visual system conditioned by the shape and the direction of movement of an object. Since these features are normally present in food observation. If those features are not present, the frog shows no interest in the visual stimulus. A similar behaviour can be observed in some kind of beetles that look for brown, big, cylindrical objects for reproductive purposes and sometimes end up mistaken other objects like brown crystal bottles with female individuals of it specie XD.
The main point of Werner’s talk was to show that our brain is not objective in the interpretation of sensory cues that it receives from the environment and for that reason our interpretation of reality and behaviour is strongly influenced by the survival mechanisms that our brain developed. In the end of the talk Werner used a genetic algorithm example to show how this interpretation of reality in terms of the survival mechanisms can help or not for an individual to survive in a given environment.
Learning in Brains and Machines by Steffen Schneider
Deep neural networks are by far the most successful machine learning models nowadays when it comes to the processing of large datasets composed of natural signals. While originally inspired by advances in neuroscience in the 1950s, the field has branched to a certain extent over the past years. The talk of our colleague Steffen Schneider addressed the core principles in training neural networks and highlighted difficulties that today’s best algorithms face when it comes to robust inference making a comparison with human visual system.
Steffen’s Talk also addressed the core problems of adversarial samples for classification systems and how to deal with domain shift under different noise models making a parallel between the way in which human brain deals with this problems in object recognition and the way we could integrate this brain solutions to artificial neural network architectures.
Sensory-motor proprioception in robotics by Germán Díez Valencia
In this talk our colleague Germán Díez Valencia showed us some highlights of his work in the emulation of sensory-motor proprioception in robots. Through his presentation we discussed important topics like predictive coding and the free energy principle, and its viability to be applied in this emulation framework.
The talk also went through concepts like sensor fusion and how to make this fusion using deep learning architectures, as well as how to use this deep learning architectures to fuse sensory cues, and how possibly one can use this to model sensory-motor proprioception in robots.
One important discussion derived from this talk was the use of deep learning architectures to model cognitive functions, as well as the connection between different variational learning architectures with the modeling of cognitive functions. For instance, the model of free energy through variational autoencoders.
Politics in Bavaria by Auguste and Jonas
Politics is a very important part of our everyday life. Since we are living in Germany and a lot of us aren’t fluent in German, Auguste and Jonas made sure to explain all the details about the new police law in Bavaria. They presented unbiased information which is very difficult to find, because the media tends to present stirred data, or over exaggerate opinions. After their presentation we split into groups and shared our personal opinion on this topic, the pros and cons we see on the ongoing debate about security in Bavaria.
Ethical Aspects of Neuroenhancement by Nicolas
The last talk, prepared by Nicolas, was about Huxleysia. Huxleysia is a democratic country that does not have natural resources and is facing an economic crisis. The government of Huxleysia believes that their real resources are in the heads of their people, and they want to improve these resources. Students were asked to form groups, and within 40 minutes of discussion, design neuroenhancement solutions that would help Huxleysia have smarter citizens and overcome the economic crisis. Afterwards each groups presented their solutions and Nicolas continued by talking about the definition of good life, and considerations while designing a technology that affects humans, such as involving the society while designing a technology for the society, caring about humanistic values, as well as thinking about the systemic sequence of the technological artifact.
Figure 1: Each group’s (and members') contribution to the presentation. (a) The histogram of the total contribution of each member, and groups. The outer numbers are the number of members in each group. (b) Each group’s location in the room (this is just a hint for the participants to find their group).
During the presentation, as a fun activity, we recorded the total time of each participants contribution in the presentation. The results are shown in figure 1. Some interesting information could be gained from this data. For instance, as we can see from Figure 1a, some groups have done a great job distributing the presentation throughout their members, as oppose to some other groups - whether that is an strategy or not, is left for the groups and their members to discuss.
Future Projects
Our group is very enthusiastic and productive which is the main reason why we decided to organize some extracurricular activities by ourselves. During the workshop, all of the students had a chance to put up their ideas on a list which would be discussed at the end. A lot of creative ideas were put on this list, and at the end 4 big projects came up.
Blog on Learning in Brains and Machines
One of the the interesting projects we decided to start in our retreat is a scientific divulgation Blog called Brains-Machines. Through this blog we intend to share our experience and knowledge in order to create a community around this neuroengineering related disciplines and promote synergy between multiple human sciences.
The topics we want to discuss In our Brain-Machines are:
- History / From Cybernetics to AI
- Adversarial Examples
- Robustness and Domain Shift
- Reinforcement Learning
- Biologically Plausible Learning
- Population Dynamics
- Free Energy Principle, Predictive Processing
- Approximate Inference: Variational Inference vs. Sampling
- Spiking Neural Networks
- Computational Psychiatry
- Modeling Brain Diseases
- Learning in Model Systems/Model Organisms
- Multi-Scale Modeling with Neuron Simulators
- Dynamical Systems
- Von Neumann vs. Neuromorphic Systems
- Classical Approaches & Principles to Machine Learning
- Intro to Neural Systems
- Connectomics
- Affect and Emotion in Brains & Machines
The Brain-Machines blog is thought to be a space for debate and discussion around the topics listed above. Each of those topics will constitute a section in the blog and in each section we will share articles, start debates about those topics and the relation of those topics with other human sciences and share our work through tutorials and examples.
One important aspect of the bolg is to show how the disciplines related to neuroengineering can cooperate with other sciences learning from them and as well helping them to expand their frontiers. One clear example is the latent synergy between machine learning, robotics and neuroscience, where models from them three can have a strong impact in the other sciences.
In order to create our first Blog release we plan to start with a reduced list of topics we want to talk about and continue growing till we cover the whole initial list and extend our blog to a bigger list of topics. The first topics we will cover are:
- Reinforcement Learning
- Biologically Plausible Learning
- Free Energy / PP / Active Inference
- Approximate Inference
- Dynamical Systems
We expect to have our first release in the next weeks so keep in contact and we hope you will be part of this community of knowledge :)
Cybathlon
One of interesting projects we intend to develop in the next months is to create a students team in order to participate in the Cybathlon 2020. The Cybathlon is a competition organized by the ETH Zürich that through different categories aims to develop systems that help people with disabilities using the latest assistance systems.
Cybathlon 2020 will take place at the SWISS Arena in Kloten near Zurich from 2–3 May 2020 having teams participating in six different disciplines:
- Brain-Computer interface race
- Functional stimulation bike race
- Powered leg prosthesis race
- Powered leg prosthesis race
- Powered exoskeleton race
- Powered Wheelchair race
Some of our colleagues in the master program already started to organize a group in order to participate in the Cybathlon 2020 in behalf of the technical university of Munich and the Master program in neuroengineering. They will decide the categories they will participate and the best strategy to follow to represent well our university and our study program. Continue reading our blog and we will keep you updated about the evolution of this amazing project.
Entrepreneurship
One of the important aspects of our field (neuroengineering) is to help the transformation of neuroscientific findings to neurotechnologies that could ultimately improve the quality of life for animals. Therefore, an entrepreneurship group was formed to create a more systematic approach and a entrepreneurial atmosphere for learning about this transformation together. There are quite few different topics is interested to learn about, including:
- Startups in Munich involving Neurotechnology
- Basics of Management
- Pitch Deck Creation
- How to create a startup/business in Germany/Munich
- Intellectual Property
- How to get initial funding, how to pitch to VCs (Venture Capital), seeding rounds, etc.
- What differentiates successful vs failing organizations
- Opportunities at TUM and Bavaria for startup support
- Finding a team for your idea
- Timeline of a startup
The group members meet weekly to discuss concepts and new ideas related to related the topics mentioned above.
Neuroengineering Journal Club
Science communication is an essential aspect of science which focuses on informing the public/researchers about scientific advancements, as well as invoking political and ethical thinking. As a part of neuroengineering curriculum, we also learn and discuss about ways to effectively communicate our science among ourselves, as well as to public. Neuroengineering Journal Club project is initiated to take this to a pragmatic level. The main objective of this journal club is to inspire and inform others about field of neuroengineering and interesting scientific findings related to this field in an informal and casual setting. You will hear more about this event in our future blog posts.
The Organizers
We would like to say thanks to our friends Annika Thierfelder, Francisco Zurita and Jonas Braun who made this retreat possible. They were in charge of the entire organization of the retreat, which included accommodation, transportation, guest talks and activities. They did a wonderful job coordinating people, and invested a lot of time to make this experience unique. This wonderful occasion allowed for the two batches of neuroengineering students to get to know each other better, as well as getting involved with the invited guests, maximizing our learning, teamwork and fun. We wish to express all our gratitude! You made this unforgettable experience possible!
Viktorija Dimova
Mohammad Bashiri
German Diez Valencia