NeuroOn: a case of startup marketing versus open notebook science
| Description | The log of an amateur neuroscience enthusiast and a professional researcher trying to validate the startup's claims with an open notebook experiment - full recording of the analysis, together with Jupyter Notebook and all the signal files, is available online. |
|---|---|
| Slides | https://alxd.org/slides/neuroon-lightning-talk.pdf |
| Website(s) | https://alxd.org/neuroon-analysis-sources.html |
| Tags | science, open notebook, neuroscience, jupyter, eeg, python, scipy, numpy |
| Person organizing | Alxd |
| Contact: | alxd@alxd.org |
| Language | en - English |
| Duration | 5 |
| Desired session | Day 2 |
| Desired timeframe | begin |
Full description of a talk I proposed for 33C3 but sadly wasn't accepted:
In December 2013 the startup "Intelclinic" announced their first product - the "NeuroOn" sleep mask, which promised polyphasic sleep assistance and sleep stage analysis using only three electrodes located on the forehead. With no whitepapers or patents to back up their claims, they encountered criticism from several neuroscience popularizers, what did not stop them from achieving commercial success on Kickstarter.
After more than two years in development, pivoting and abandoning their original claims, Intelclinic agreed to lend Pawel 'alxd' Chojnacki a test unit of the NeuroOn for review. He decided to create a full open notebook research of the mask, comparing its capabilities to a professional polysomnograph in a hospital laboratory with the help of Ryszard Cetnarski. The full recording of our analysis, together with Jupyter Notebook and all the signal files, is available online.
We'd like to share our story as a neuroscience enthusiast and a professional researcher, and tell you more about our findings and the process of analysing the mask. alxd can share the good practices which helped us to finish the project, and the problems which led him to ask a professional for help. Ryszard will be able to tell you more about the analytic tools we used and the science behind the experiment.
With startups making more unfounded claims and Academia becoming less transparent every day, there is a growing value in clear and verifiable research, available for everyone to criticize and comment on. We often forget that not every publication has to be a breakthrough to matter, and current incentive systems tend to award shady manipulations of data.
Our experiment is not only the analysis of a single well-marketed device, but also a commentary on many trends in modern scientific research.
