Application of mixed reality technique in oral and maxillofacial surgery

doi:10.19438/j.cjoms.2020.06.017

Abstract

Abstract: Mixed reality (MR) technique is a new type of holographic imaging technique that appears after virtual reality and augmented reality, and has been widely valued in the field of oral and maxillofacial surgery. The technical characteristics of MR appeared more advantages than previous three-dimensional visualization techniques. MR technique has showed broad development prospect in operation planning, intraoperative navigation, training of operation and other aspects. However, some limitations are displayed during the application of MR technique in clinic, which mainly reflected in the medical image processing system, image registration accuracy, positioning system stability, and human-computer interaction experience. Therefore, breaking the bottleneck of MR technique and improving its clinical adaptation are the main directions for the development of MR technique in oral and maxillofacial surgery.

Key words: Mixed reality, Oral and maxillofacial surgery, Visualization, Digital surgery, Human-computer interaction

CLC Number:

R782

CHEN Ming-tao, YANG Rong, JI Tong. Application of mixed reality technique in oral and maxillofacial surgery[J]. China Journal of Oral and Maxillofacial Surgery, 2020, 18(6): 559-562.

References

[1] Milgram P, Kishino F.A taxonomy of mixed reality visual displays[J]. IEICE Transact Inform Syst, 1994, 77(12): 1321-1329.
[2] Azuma R, Baillot Y, Behringer R, et al.Recent advances in augmented reality[J]. IEEE Comput Graph Appl, 2001, 21(6): 34-47.
[3] 黄进, 韩冬奇,陈毅能, 等. 混合现实中的人机交互综述[J]. 计算机辅助设计与图形学学报, 2016, 28(6): 869-880.
[4] Cho KH, Papay FA, Yanof J, et al. Mixed reality and 3D printed models for planning and execution of face transplantation[J]. Ann Surg, 2020 Mar 27 [Online ahead of print].
[5] Mohammad AG, Mehrdad H, Mehrdad N, et al.Augmented reality for breast tumors visualization[J]. Conf Proc IEEE Eng Med Biol Soc, 2016, 2016: 4391-4394.
[6] Saito Y, Sugimoto M, Imura S, et al.Intraoperative 3D Hologram support with mixed reality techniques in liver surgery[J]. Ann Surg, 2020, 271(1): e4-e7.
[7] Gao Y, Tan K, Sun J, et al.Application of mixed reality technology in visualization of medical operations[J]. Chin Med Sci J, 2019, 34(2): 103-109.
[8] Chen XJ, Xu L, Wang YP, et al.Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted display[J]. J Biomed Inform, 2015, 55:124-131.
[9] Ewurum CH, Guo YY, Pagnha S, et al.Surgical navigation in orthopedics: workflow and system review[J]. Adv Exp Med Biol, 2018, 1093: 47-63.
[10] Kubicek J, Tomanec F, Cerny M, et al.Recent trends, technical concepts and components of computer-assisted orthopedic surgery systems: a comprehensive review[J]. Sensors (Basel), 2019, 19(23): 5199.
[11] Bosc R, Fitoussi A, Hersant B, et al.Intraoperative augmented reality with heads-up displays in maxillofacial surgery: a systematic review of the literature and a classification of relevant technologies[J]. Int J Oral Maxillofac Surg, 2019, 48(1): 132-139.
[12] Zhu M, Liu F, Chai G, et al.A novel augmented reality system for displaying inferior alveolar nerve bundles in maxillofacial surgery[J] Sci Rep, 2017, 7: 42365.
[13] Rozen WM, Paddle AM, Chubb D, et al.Guiding local perforator flaps with preoperative imaging: revealing perforator anatomy to improve flap design[J]. Plast Reconstr Surg, 2012,130(1): 130-134.
[14] Jiang TR, Yu DW, Wang YQ, et al.HoloLens-based vascular localization system: precision evaluation study with a three-dimensional printed model[J]. J Med Internet Res, 2020, 22(4): e16852.
[15] Sappenfield JW, Smith WB, Cooper LA, et al.Visualization improves supraclavicular access to the subclavian vein in a mixed reality simulator[J]. Anesth Analg, 2018, 127(1): 83-89.
[16] Mitsuno D, Hirota Y, Akamatsu J, et al.Telementoring demonstration in craniofacial surgery with HoloLens, Skype, and three-layer facial models[J]. J Craniofac Surg, 2019, 30(1): 28-32.
[17] Condino S, Turini G, Parchi PD, et al.How to build a patient-specific hybrd simulator for orthopaedic open surgery: benefits and limits of mixed-reality using the Microsoft HoloLens[J]. J Healthc Eng, 2018, 2018: 5435097.
[18] Tejo-Otero A, Buj-Corral I, Fenollosa-Artés F.3D printing in medicine for preoperative surgical planning: a review[J]. Ann Biomed Eng, 2020, 48(2): 536-555.
[19] Jud L, Fotouhi J, Andronic O, et al.Applicability of augmented reality in orthopedic surgery-a systematic review[J]. BMC Musculoskelet Disord, 2020, 21(1): 103-108.
[20] Tan WJ, Ge W, Hang YC.Computer assisted system for precise lung surgery based on medical image computing and mixed reality[J]. Health Inf Sci Syst, 2018, 6(1): 10-18.
[21] 武雪梅, 刘奇. 基于立体视觉的手术导航光学定位系统[J]. 中国组织工程研究与临床康复, 2011, 15(35): 6584-6587.
[22] 贺长宇, 刘越, 王涌天. 用于增强现实手术导航系统的光学-惯性混合跟踪方法[J]. 计算机辅助设计与图形学学报, 2016, 28(3): 513-519.
[23] Groves LA, Carnahan P, Allen DR, et al.Accuracy assessment for the co-registration between optical and VIVE head-mounted display tracking[J]. Int J Comput Assist Radiol Surg, 2019, 14(7): 1207-1215.
[24] Wang JC, Shen Y, Yang S.A practical marker-less image registration method for augmented reality oral and maxillofacial surgery[J]. Int J Comput Assist Radiol Surg, 2019, 14(5): 763-773.
[25] Incekara F, Smits M, Dirven C, et al.Clinical feasibility of a wearable mixed-reality device in neurosurgery[J]. World Neurosurg, 2018, 118: e422-e427.
[26] Lee C, Rincon GA, Meyer G, et al.The effects of visual realism on search tasks in mixed reality simulation[J]. IEEE Trans Vis Comput Graph, 2013, 19(4): 547-556.
[27] Pepe A, Trotta GF, Mohr-Ziak P.A marker-less registration approach for mixed reality-aided maxillofacial surgery: a pilot evaluation[J]. J Digit Imaging, 2019, 32(6): 1008-1018.
[28] Mitsuno D, Ueda K, Hirota Y.Effective application of mixed reality device HoloLens: simple manual alignment of surgical field and holograms[J]. Plast Reconstr Surg, 2019, 143(2): 647-651.
[29] Jiang TR, Zhu M, Chai G, et al.Precision of a novel craniofacial surgical navigation system based on augmented reality using an occlusal splint as a registration strategy[J]. Sci Rep, 2019, 9(1): 501-508.
[30] Tepper OM, Rudy HL, Lefkowitz A.Mixed reality with HoloLens: where virtual reality meets augmented reality in the operating room[J]. Plast Reconstr Surg, 2017, 140(5): 1066-1070.
[31] Vaughan N, Dubey VN, Wainwright TW, et al.A review of virtual reality based training simulators for orthopaedic surgery[J]. Med Eng Phys, 2016, 38(2): 59-71.
[32] Huber T, Hadzijusufovic E, Hansen C, et al.Head-mounted mixed-reality technology during robotic-assisted transanal total mesorectal excision[J]. Dis Colon Rectum, 2019, 62(2): 258-261.
[33] Lei PF, Su SL, Kong LY, et al.Mixed reality combined with three-dimensional printing technology in total hip arthroplasty: an updated review with a preliminary case presentation[J]. Orthop Surg, 2019, 11(5): 914-920.