While there are many ways in which medical science is using digital techniques and artificial intelligence (AI) to make life-saving decisions., the science and art of animation holds the potential to help revolutionize healthcare. Once the province of movies and cartoons, more realistic computer-generated imagery in medical science is providing solace to patients undergoing harsh treatments. And at the meeting point of movies and medicine, doctors are using digital technology to help us deal with pain, deliver rationalized dosages, and help people in remote areas get timely medical assistance.
The eyes crinkle, irises the size of tiny pinpricks, and a single tear rolls down her eyes, washing away sadness. The princess’s eyes widen like pools, threatening to overflow any minute. We’ve all been there, watching a Disney animation, crying with Moana (2016) when her grandmother passes away, leaving behind secrets for the little one, or when a magical tear-drop from Rapunzel (2010) brings her loved one back.
Making life-like animation – getting the eyes right, in particular, so that computer generated imagery (CGI) makes a character’s pupils expand and contract of their own accord as lighting changes in a scene – complete with a storyboard and script, is a place where movies meet medical science.
For the right animation, every detail is recorded – the white sclera, the transparent cornea, and the colored iris, detailed vein texture, and even exact curvature along with the refractive index at the boundary of the cornea.
Elaborating on how eyes are unique and contain details and “imperfections”, a Disney Research Zurich paper authored in 2014 by Pascal Berard et al, titled High-Quality Capture of Eyes, said: “The micro-geometry of the iris is as unique to every person as a fingerprint, and its position and deformation depends on the accommodation of the underlying lens.”
Animators who work on creating life-like eyes that make the audiences tear-up, often see their work acquire significance for ophthalmologists, who can use similar technology to detect illness. CGI intersects with the medical world, proving that science fiction is not the only meeting point between the vastly different worlds of movies and medicine.
Clinically speaking, using the same technique to model the eye in three dimensions by using a single scanned image of the face can help medical technicians assess any changes over time since change in the eye is often a sign of disease.
At the World Government Summit (WGS) in Dubai on February 11-13, 2018, eminent researchers and scientists present the latest developments taking place in the field of digital medicine. John Cryan, Professor and Chair, Department of Anatomy & Neuroscience – University College Cork, delivers a talk on ‘A Gut Feeling about Happiness’. Also, James Doty, Clinical Professor of Neurosurgery, Stanford University, speaks on ‘Solving Mysteries of the Brain and Secrets of the Heart’.
ASSISTING THE DOCTORS IN HEALING BETTER
There are many ways in which medical science is using digital techniques and artificial intelligence (AI) to make life-saving decisions. American space agency NASA is working with a medical company to create (ro)bots weighing 0.4 kg that can be placed inside a patient’s body and can be remotely controlled by a surgeon on Earth so procedures can be performed in space – bringing to mind the interrogation in Matrix where Agent Smith brings out a bot that burrows into Neo’s belly button.
The bot in your belly may be on to something more than just invasive surgery since there is much worth peeking into. At his session on gut health at the WGS, Professor Cryan said: “Trillions of microbes that reside in your gut play a role in programming health. There is a reduction in microbes in the gut as a response to stress. What we do know is that microbes are little factories producing weird and wonderful chemicals that support our health.”
When outside, the robots are hard at work. Already, the whimsically named da Vinci surgical robot complements surgeons with human wrist-like movement via its instruments and offers 10 times enlarged 3D views. It is extensively used with four million procedures worldwide already having been carried out.
In another part of the world, researchers at the Singapore National Eye Center have built AI that can instantaneously screen retinal images with or without diabetic retinopathy, glaucoma and age-related macular degeneration, dramatically reducing the workload of optometrists, freeing up their time for treatment and care.
At the John Radcliffe Hospital in Oxford, England, researchers have developed an AI diagnostics system that can predict the risk for heart disease much earlier than is done now, thereby allowing doctors time to contain it before it develops further. If findings are confirmed later this year, the system will be made available to the UK’s National Health Service (NHS) for free.
Microbiologists at Harvard University have used AI to create a smart microscope to detect deadly blood infections after being trained on 100,000 images culled out of 25,000 slides.
Researchers in Showa University in Yokohama, Japan have demonstrated an AI-based endoscopic system that can identify colon cancer in less than a second.
Watson – International Business Machines Corp’s (IBM) question answering supercomputer combing AI and analysis, named after the company’s founder – uses deep learning techniques to provide clinicians worldwide with evidence-based treatment plans.
Developed and “trained” by Memorial Sloan Kettering Cancer Centre in New York City, IBM Watson for Oncology provide suggestions backed by medical literature, data, and case studies, allowing doctors to look beyond their immediate experience and tap into the global knowledge on developing treatment plans for breast, lung, colorectal, cervical, ovarian, gastric, and prostate cancers. Already, over 40 hospitals in China are deploying Watson for help.
Watson is not IBM’s first brush with a personal computer with a name. When Stanley Kubrick made his movie 2001: A Space Odyssey, he reportedly went to IBM for drawings and blueprints that could imagine the future of personal computing and was told that an intelligent machine of this magnitude would be something into which men went, rather than walked around.
In reality, artificial intelligence is playing a complementary role closer to what is shown in movies such as Elysium (2013), where a multi-functional radiology machine checks you and cures you in minutes, or Cloud Atlas (2012), which shows a handheld medical scanning device in action, which can analyze, detect and cure diseases.
Directly referencing Star Trek, where devices called Tricorders are used, the highly respected Qualcomm Tricorder X Prize is given for developing a device that will “accurately diagnose 13 health conditions and capture five real-time health vital signs, independent of a health care worker or facility, and in a way that provides a compelling consumer experience.”
Experts such as Bertalan Meskó, whom some call the Medical Futurist, say Watson is “the stethoscope of the 21st century”. The role of AI and other technologies does not replace the physician, but instead complements their work in clinical care and provides diagnosis quickly and accurately.
MOVIES MEET MEDICINE
Movies and medical science interact in miraculous ways, often allowing the medical community to visualize and reach where human eyes and hands cannot. It can also help transport patients to a better world. In rural Mexico, Dr. Jose Luis Mosso Vazquez uses virtual reality (VR) instead of sedatives during surgery. Having watched his son become so engrossed in a game that he was oblivious to the world around him, he decided to try using VR so patients could escape to their reality of choice away from the operating table.
There is already proof of concept for this. A controlled study carried out during esophageal, stomach, and duodenum endoscopy found: “Overall pain was significantly lower in the VR group than the control group with a moderate effect size. Physician stress was also reduced in the VR group, allowing greater accuracy and a shorter procedure time.”
Movies transport us into another reality, using real actors. Sometimes there are limitations, requiring technology to come to their aid. Filmmakers used artificial intelligence to recreate digital versions of the actor Paul Walker after his death in 2013, in order to complete Furious 7. Fast-forwarded into the future, in the movie Guardians of the Galaxy Vol. 2, technology combines the footage of current-day Kurt Russell with reference shots of his younger self to make the character look younger when the script demanded it.
For doctors, medical animation helps break down concepts so that they can be applied more easily. An application of life-like, but not life-size visualizations was seen recently in Crispr gene-editing software. On January 28, 2018, Visual Science and Skoltech Biotechnology faculty created a non-profit scientifically accurate 3D animation of Crispr based on real molecular structures. Jennifer Doudna, Professor of the Departments of Molecular and Cell Biology and Chemistry at the UC Berkeley, Executive Director of the Innovative Genomics Institute, says: “Molecular animations are an essential way to demystify and explain complex biological systems.”
When truly personalized, this technology can be used for doctor’s aid in helping each patient. One example is in True 3D medical visualization EchoPixel, which renders patient-specific anatomy in an intuitive, interactive, virtual reality format. It allows physicians an opportunity to view and interact with patient tissues and organs in a truly 3D form, as if they were real physical objects, and navigate around the patient’s body in virtual reality.
Also published on Medium.