We all have dreamt of flying, fighting with a lightsabre, and controlling objects with our mind. I was lucky enough to make one of my dreams come true when DigiLab UK went on an exploration journey of brain-computer interfaces. I recruited one fellow dreamer, a UX designer, along with me, the software engineer. We started to look at different aspects of BCI. The initial task chosen was to control an object with our mind, and in the journey, learn more about the technology. I was staring at my desk thinking about which object to control. Then there was my answer staring back at me, BB8 on my desk. Whether by fate or the force, we knew what we had to do. We would control BB8 using a BCI device, the Emotiv EPOC+, which was also available and previously used for hackathon project in Norway. I will take you through my journey of making this prototype with the help of a two-part series blog in the hopes of helping others who are starting to explore BCI technology.
The Emotiv EPOC+ headset comes along with 14 electrodes. Setup of the device is easy but tedious as you are required to soak the electrodes with saline solution each time before screwing them onto the device. This process is needed to get good connectivity between the user’s scalp and electrodes. For people with more hair, it is naturally more difficult to get good connectivity as they must adjust their hair to make sure there is nothing between the electrodes and scalp. For some connectivity levels were sufficient with dry electrodes but to save time I recommend that always soak the electrodes before using the device as you are more likely to get fast and good connectivity. There are many videos available online that guide you through the initial setup of the device.
Training mental commands
I aimed to control BB8 with EPOC+ headset, so I started to investigate the mental commands and its various functionalities. To use the mental commands you first need to train them. The training process enables the EPOC+ to analyze individual brainwaves and develop a personalized signature corresponding to the different mental action.
Emotiv Xavier control panel is an application that configures and demonstrates the Emotiv detection suites. It provides the user with an interface to train mental commands, view facial expressions, performance metric, raw data, and to upload data to Emotiv account. The user has the option to sign in to their account or use the application as a guest.
The user is required to make a training profile. Users have the option to have multiple training profiles under one Emotiv account. Each user needs their profile as each one of us possesses unique brain waves.
Let’s train the commands
The first mental command or action user must record is their “neutral” state. The neutral state is like a baseline or passive mental command. While recording this state, it is advisable to remain relaxed like when you are reading or watching TV. If the neutral state has not recorded correctly, the user will not be able to get any other mental commands working properly. For some recording, the neutral state results in better detection of other mental commands.
The “record neutral”button allows the user to record up to 30 seconds of neutral training data. The recording automatically finishes after 30 seconds, but the user has the option to stop recording any time they feel that enough data has been collected. At least 6 seconds of recorded data is required to update the signature.
After recording the neutral state, the user can start to train any one of the 13 different actions available. For my research, I only focused on two mental actions “push” and “pull.” Emotiv website provides tips and instruction on how to train the mental commands. It suggests remaining consistent in thoughts while training. To perform any mental action, users must replicate their exact thoughts process or mental state that they had during the training process. For example, if a user wants to train “push” command, it’s up to the user what they want to think or visualized for that action. Some users might imagine a cube going away from them, or some might imagine a cube shrinking, whatever works for them, but they need to remain consistent in their thoughts and mental state. If the user is distracted even for a second, it is advisable to retrain the action. As the user is able to train a distinct and reproducible mental state for each action, the detection of these actions become more precise. Mostly, the users must train an action several times before getting accurate results.
While I was trying to train the “push” action, I placed the BB8 on a white table and imagined it moving away from me. I replicated same thought, imagining BB8 going away from me on the table and was able to perform the mental action. However, when I placed the BB8 on the carpet, I failed. This may have been because the different colour of the carpet distracted me and I was unable to replicate my exact mental state, therefore, failed to perform the mental action. For me, the environment needed to be the same to reproduce my specific mental state. However, this varies from user to user.
Emotiv Xavier gives the option to view an animated 3D cube on the screen while training an action. Some users find it easier to maintain the necessary focus and consistency if the cube is automatically animated to perform the intended action as a visualization aid during the training process. A user can, therefore view themselves performing an action by viewing the cube. The cube remains stationary unless the user is performing one of the mental actions (if already trained) or unless the user selects “Animate model according to training action” checkbox for training purposes. It is advisable to train one action fully before moving on to the next one. It gets harder and harder to train as you add more mental actions.
Is the training process easy?
There are lots of tips and guidance given on Emotiv website for training mental commands. Users are given an interface to help them train and perform mental actions with the aid of animated 3D or 2D models. However, during my three days of training, I was not able to find an easy and generic way to train the mental commands. People are different. Some are more focused than others. Some like to close their eyes to visualize and perform the command. Some want help with animation. What I observed was that it depends on the person and how focused they are, and how readily they can replicate a state of mind. There is no straightforward equation. You need time and patience. I was only able to achieve 15 % skill rating after training two mental actions. Only one of my colleagues got 70% skill rating which he wasn’t able to reproduce later.
While searching for simpler ways to train mental commands I came across a process known as neurofeedback. Neurofeedback is a procedure for observing your brain activity to understand and train your brain. A user observes what their brain is actually doing as compared to what they want it to be doing. The user monitors their brain waves, and if they are nearing the desired mental state, then they are rewarded with a positive response which can be music, video or advancing in a game. Neurofeedback is used to help reduce stress, anxiety, aid in sleeping, and for other forms of therapeutic assistance.
Neurofeedback is a great way to train your brain for mental commands. For example, if someone is trying to do “push” command,” they can observe their brain activities on screen and see if they are consistent. Then they can slowly and steadily train their brain to replicate a specific state. Emotiv provides the “Emotive 3D Brain Activity Map” and “Emotiv Brain Activity Map”, a paid application that can be used to monitor, visualize and adjust brainwaves in real time. For our research, we didn’t try these applications. If you try it out, let us know how you got on!
Training is like developing a new skill. Remember how you learned to ride a bike, or how you learned to drive? It took time and practice, and it’s the same for training mental commands. Companies do provide help by giving tips, instruction and software applications to help users train and visualize, but in the end, it’s acquiring a new skill, and users need practice. Some might learn faster than others, but for everyone it takes time.