Towards reproducible science with Jupyter Notebooks
Computed using Ubermag by Martin Lang, Marijan Beg and Hans Fangohr
For publications, the scientific principle of reproducibility requires that studies and processes are documented and archived to such a level that they can be repeated at a later point - by the original authors as well as by other interested researchers. Technically, this should be possible for the software-driven steps. However, in practice reproducibility is often not given - for a multitude of reasons.
Now researchers from the UK, France, Norway and Germany - led by Hans Fangohr from the MPSD’s new Computational Science unit - have published practical solutions for this challenge on the basis of the OpenDreamKit research project. They recommend reproducible workflows that include the use of open source software and the provision of complex computational capabilities through researcher-friendly high-level command interfaces. These allow scientists to direct the research via electronic notebooks which automatically record every step that has been taken.
One of the open source results of this work is the Ubermag simulation environment for magnetism at the nanoscale. The low-level simulation software, that has been optimized for high execution performance, is difficult to use directly, and the researchers developed a high-level Python interface to drive the simulations with much greater ease. The Ubermag environment features visualisation capabilities which provide immediate visual feedback on the simulation results and integrate naturally into the Jupyter notebook. Given this tool set, a simulation study can now be driven from the Jupyter notebook, which combines narrative comments by the researcher with commands to execute the simulation, immediately followed by the results obtained. The notebook has been designed to be able to record and archive this rich and executable document - thus providing a leap forward for the reproducibility of the study.
"To improve reproducibility in science, it is not enough to provide tools or workflows that make the results more reproducible. These tools must also fit into the scientific approach of the researchers and ideally lead to a better research experience, so that the new methods are accepted and adopted widely," says Hans Fangohr, coordinating author of the publication.
Reproducibility is attracting increasing attention from researchers, research councils and publishers. "One additional aspect of the discussion is that reproducibility leads to re-usability,“ Fangohr explains. "If all the steps of a data analysis are documented and archived - ideally in a (potentially long!) computer-executable script - so that they can be repeated automatically at the touch of a button, then it becomes a lot easier to take such a study and extend it.
"Currently, it is not unusual for PhD students to spend months or years to first reproduce published results as a baseline, before they can extend the work with novel additions as the second step. With the workflows and methods proposed here, we can reduce that time to hours or minutes, making our collective research activity a lot more efficient."
Other news from the department science
Replicating the structure of bird feathers
The new material could be used in batteries or filtration
Quantum tool opens door to uncharted phenomena
Method can contribute to a better understanding of quantum materials
Recovering instead of shredding: recycling batteries more efficiently
KIT researchers are working with industry to develop a more sustainable recycling process to recycle materials from lithium-ion batteries more effectively
Industry 4.0: No impact on energy consumption?
To what extent does the digitalisation of industrial and manufacturing processes (Industry 4.0) improve energy efficiency and thus reduce energy intensity?
New approach to the sensible utilisation of carbon dioxide from car exhaust gases
"A method has been discovered that uses impure CO2 streams and enables a breakthrough in the synthesis of valuable chemicals and pharmaceuticals"
Boosting PET recycling with higher standards for laboratory experiments
New study shows how enzymatic plastic degradation could be brought one step closer to commercialisation
Innovating Optoelectronic Components with Phosphorus
Significant breakthrough: phosphorus chemists develop new method to selectively introduce phosphorus and nitrogen atoms into polyaromatic systems
Artificial intelligence finds ways to develop new drugs
The chemists tested the process using borylation – a reaction that activates hydrocarbon scaffolds
X-rays reveal how glasses lose their stability
PETRA III experiment shows how atoms in glass behave as weaknesses appear
Understanding the strength development mechanism of chemically treated sandy soil
Researchers provide insights into how the chemical injection process increases soil strength, paving the way to advancement of next-generation construction
Researchers obtain promising results against capacity loss in vanadium batteries
A computational study conducted in Brazil could help extend the working lives of these batteries, which are widely used by utilities and manufacturers
Making better use of residual gases
Award for invention for environmentally friendly adsorption of chlorine
Most read news
Plastic-eating bacteria turn waste into useful starting materials for other products
Microbial Upcycling of Waste PET
Microbes could help reduce the need for chemical fertilizers
A coating protects nitrogen-fixing bacteria: Start-up to commercialise coated bacteria for large-scale use in regenerative agriculture
New designs for solid-state electrolytes may soon revolutionize the battery industry
Scientists achieve monumental improvements in lithium-metal-chloride solid-state electrolytes
Dow and Evonik announce startup of hydrogen peroxide to propylene glycol (HPPG) pilot plant
Innovative technology offer flexibility, lower costs, and a smaller environmental footprint
This is a battery
Two colored liquids bubbling through tubes: Is this what the battery of the future looks like?
Inauguration of the world’s first pilot plant for the cost-efficient production of green methanol
Start-up C1 Green Chemicals AG and research partners develop fundamentally new production process
Converting PFAS “forever chemicals” into valuable compounds
Scientists develop a new method to incorporate harmful perfluoroalkenes into N-heterocyclic carbene ligands
Graphene's proton permeability: A switch for future energy technologies
This discovery could lead to the development of more efficient hydrogen fuel cells and solar water-splitting devices
Lithium-ion batteries are no longer the gold standard in battery tech
On the way to safer and more powerful energy sources
Not so silver lining: Microplastics found in clouds could affect the weather
Low-altitude and denser clouds contained greater amounts of microplastics
CO2-free hydrogen: BASF receives funding approval for 54-megawatt water electrolysis plant
Proton exchange membrane (PEM) electrolyzer expected to produce up to 8,000 metric tons of hydrogen per year
More news from our other portals
Bowel cancer: aspirin activates protective genes
Researchers have identified a signaling pathway by which aspirin can inhibit colorectal cancer.
Dunning-Kruger effect with muesli bars
Those who know the least consider themselves highly competent
Autonomous measuring instruments systematically detect new materials
A new algorithm measures materials libraries up to four times faster than before: It’s based on machine learning
New drug delivery system could reduce daily diabetes shots to just three a year
Dietary management drugs have transformed Type 2 diabetes care, but daily injection routines are challenging for some patients
Naked Clams: The New Superfood Sensation Emerging from the Depths
Researchers found Naked Clams contain almost twice the amount of Vitamin B12 as blue mussels and have developed an efficient way to farm them
Aston University technology to combat the not-so sweet practice of honey fraud
Light technology to be used to detect if honey is blended with cheap additions
Pushers, overcrowded trains and phone zombies
Sprite presents the world's first vending machine that responds to the things that bother Generation Z the most
Researchers discover new ultra strong material for microchip sensors
A material that doesn't just rival the strength of diamonds and graphene, but boasts a yield strength 10 times greater than Kevlar
How stem cells and immune cells communicate
Lisec Artz Award for Simon Haas: Groundbreaking discovery of an unknown protective mechanism against blood cancer from stem cells
Viral Impostors: Breakthrough for Virus Research
The penetration of viruses into cells can now be tracked with unprecedented accuracy thanks to innovative design for pseudoviruses
From the trough to the plate - digitally calculated
Computer program "ConTrans" estimates how much of an undesirable substance is transferred from animal feed to food
Fatty acid factory filmed at work
High-resolution images provide new insights into cellular fatty acid production: Potential for medicine and biotechnology
Tracking down Environmental Toxins
Detection of per- and polyfluoroalkyl substances (PFAS) by interrupted energy transfer
The weight of pollution: exposure linked to obesity
Chronic exposure to environmental pollutants found to increase risk of cardiovascular disease
Tönnies Group launches first nationwide "Meat Climate Platform"
100 guests at the Future Forum for Agriculture
Scientists use quantum biology, AI to sharpen genome editing tool
"This study represents an exciting advancement toward, understanding how we can avoid making costly ‘typos’ in an organism’s genetic code"
Textbook knowledge turned on its head: 3-in-1 microorganism discovered
Newly multifunctional bacterial species
Could eating turkey ease colitis?
According to data in mice, extra tryptophan could reduce the risk of future colitis flares