Saturday, 16 September 2017

Digital 3D Orthognathic Surgery with the OrtogOnBlender addon

Timelapse video of orthognatic planning steps

The first time I understood the concept of programming, it was in the year 2001, when I realized that a text with some codes was translated into a three-dimensional scene with some animated elements. I was astounded by it, as when I was three years old I realized that the water that was coming into the refrigerator freezer were those transparent pebbles my grandparents used in the juices they prepared for the family.

Since then I have always consumed some information about programming. Sometimes it would even cheer me up and write some codes, but the lack of practical use made me confine myself to just exercising my brain to understand some abstractions that involved that reality.


Many years and many books later, here I start teaching 3D computer graphics courses for the health area and realize that the organization of Blender graphic interface more confused than it helped the doctors in their didactic absorptions.

Faced with this difficulty on the part of the students, I motivated myself to create a methodology that would facilitate students' understanding in a field that proved to be quite challenging in the didactic context: orthognathic 3d digital surgery planning.

Automatically created deformation area

Unlike the other courses I had taught so far, orthognathic surgery, when taught in the "manual mode," where the student needs to understand detail by detail of commands, is almost impossible to apply to a short course. The problem is that it is precisely short courses that are the most sought after and not offering them translates into not generating income.

Together with Dr. Everton da Rosa, Dentist, orthognathic surgery specialist, I began to develop a series of small scripts that would help students automate one task or another. I did one, it worked, I cheered up... I did another, it worked out and another... I picked up a little and kept staggering and achieving, although not in the most elegant way, at least it was functional.

Jaw rotation

Blender uses Python for its scripts and this language is marvelous for anyone who wants to try to program, since the code is pretty clean. My excitement was growing to the point that I was fissured by the small challenges that were appearing.

Initially I intended to create a series of scripts, but realized that I could bundle everything into an addon, which would be installable in Blender. When programming and working with different objects and modes, I was intuitively understanding how everything happened, to the point of thinking of a solution, writing without referencing and code working!

Automatic creation of vertex groups

It was what I needed to become a hermit cyber... locked in the bedroom, just thinking about how to solve the problems and expand the functionalities. In unlikely five days, though not finalized, I was able to mount an addon sufficiently functional to simplify and automate a series of steps that comprise the planning of orthognathic surgery.

Interestingly, as a good nerd who follows the best practices of self-taught effort, I did not ask anyone for anything, I did not post on forums or social media groups on programming, nor did I ask for help from friends who are masters of language. I found everything I needed in the documentation, videos on Youtube, tutorials on the internet and, pampering ... in templates provided by Blender himself! In short, I did not ask for direct help, but I got it from the work and goodwill of people who wanted to help and made excellent materials, I am very grateful to all!

Jaw after osteotomy (Cork on Blender)

I intend to make the addon available soon (as we did with Cork on Blender), after testing it with my students. It is not yet complete, since it lacks the part of the surgical splint (a sort of guide for fitting the teeth) and some other activities such as automatic parenting with bone armature.

Even so, I am very satisfied, mainly for having implemented the automatic process of creating areas of influence based on osteotomy. With it, the cut bones, when moved make the soft tissue (skin, fat, etc.) deform. You can not imagine the challenge of teaching this in the "manual mode." Luckily it all came down to clicking two buttons, which is not bad!

Deformation radiated by the skull

I'm so happy and motivated that the will it gives is to go around programming and automating everything, but I felt on the skin another situation that is inherent in this kind of knowledge ... a code needs to be elegant and be prepared for exceptions and mine does not pass nor close to it. The trend now is to lapse the code and try to make it decent, worthy of being shared, used and improved by the community and interested in using it.

That's it, now I'm going because I had an insight on how to solve a problem in one part of the code... a big hug! : D

Tuesday, 5 September 2017

"Cork on Blender" the addon that has saved lives is available to everyone!

Retrieval of low grade osteosarcoma with virtual planning of microsurgical reconstruction with fibula flaps. Credit: Bruno Aragão.
"First do, then show"... this has been the mantra of our team. Since August 2014, three years ago, great challenges have arisen when I was contacted by Dr. Everton da Rosa to teach him 3D computer graphics techniques so that he could use them in his virtual orthognathic surgery plans.

Orthognathic surgery is a technique that corrects facial deformities in adults. I myself would need this procedure, since my upper jaw is small and the mandible is projected. This makes me look not very pleasing to the eyes, but fortunately I breathe well and ... I'm married, what would be an accomplishment for an antisocial nerd ... imagine for a short, antisocial and ugly nerd!

Joke aside, many people have serious problems as a result of these deformations, which often puts the lives of these individuals at risk. When planning a surgery digitally, specialists such as Dr. da Rosa have the opportunity to approach the procedure without the risk of causing harm to the patient, and when performing the task effectively, will be very well prepared, since he did the studies several times before surgery.

Orthognathic surgery planning with Blender and Cork.
This whole digital process would work very well, if we did not have a problem on the way, this problem was Blender's native Boolean. It should be recognized that with each version the library of these calculations has become better, but when we speak of boolean for use in health sciences it presents problem at inopportune moments.

To solve this problem, still in 2014, I looked for a classic exit for those (like me) working with Linux and free software: research a project that would allow the robust calculation of Boolean. I ended up finding it in the tool Cork, one of the great works of the developer Gilbert Bernstein.


Examples of using Cork on Blender. Left: Reconstructed hull of Freddy's turtle, in the center: facial prosthesis (Credits: Mais Identidade and UNIP) and left: prosthesis of a beak.

I was delighted with the results and proceeded with the tests, although these depended on exporting the files for the calculations to be done externally. As the activity was not very practical, came into the picture the one who made use of Cork within our beloved Blender, the expert in Python Script Dalai Felinto.

He masterfully created a code that turned a series of boring activities (export, command line, etc.) into a few grouped buttons on the upper left of 3D View. This was the trigger for a number of interesting projects.

From that moment, we started to do the surgery plans almost completely within Blender. We imported the reconstructed (skin and bone) surfaces from tomography and performed the planning, now without suffering from the part of the osteotomy, the one in which we cut the bones.

With Cork inside Blender, the osteotomy, which once drove us crazy, was now just another process, being performed not only efficiently but quickly.

In the course of the planning, other projects have touched our daily life, from the maintenance of cultural heritage, to the making of facial prostheses for humans and in the field of Veterinary Medicine.

Together with the Animal Avengers group and specialists from Italy, I created prostheses for no less than 11 animals! There were four toucans, an aracari, a dog, a goose, a crow, a turtle, a parrot, and a macaw. In addition to this area, I worked with Dr. Rodrigo Salazar with the manufacture of facial prostheses for cancer victims who had one eye removed.

Over time, a lot of people started looking for our teams so we could provide training in all of these areas and we would naturally install the addon on the students' computers. In the face of the great demand and security provided by it, we finally decided to share this tool with everyone and today we have the honor to announce it available openly and for free!

For those who want to download, just access the following link:


And follow the instructions.

You can compile Cork or download one of the versions available for the three operating systems: Windows, MacOSX or Linux.

I would like to thank all those who have helped us to develop this tool, either by learning to use it or by making ourselves available to be part of the research.

A big hug!

Sunday, 3 September 2017

3D documentation of ancient millstones: preliminary tests


The traditional drawing of ancient millstones consists in a plan (eventually with shading to give a sense of three dimensions) and one or more cross sections of the object. This is not always easy because of the dimensions, weight and sometimes shape (mainly the Pre-Roman millstones are irregular and asymmetric) of this type of artefacts. Furthermore, the millstones are generally in museums or in storerooms: in these places, it is often difficult to move the objects or to have enough time for drawing quietly and checking well all the details. In short, drawing a millstone is not like drawing a sherd of pottery!
For these reasons, it could be useful applying a methodology based on the Structure from Motion (SfM) techniques in addition to the traditional drawing methods. In this post I’m going to present the preliminary results of a test aimed at the three-dimensional documentation of a fragment of an Iron Age millstone from Northern Italy (a so-called “Olynthus mill” or “Hopper rubber”).

The first step was the construction of a rectangular wooden frame made by 4 rods of different length (40, 60, 80 cm), so that it’s possible to build frames of different areas according to the dimensions of the millstone to be drawn. On the surface of the rods some cross marks equally spaced are signed: these marks will be use as reference points with known coordinates for the rectification of the3D point cloud, mesh and (eventually) texture (something like GCP, Ground Control Points).


Four bolts adjustable for height hold the frame together and allow to level it perfectly. Once the frame is ready, you need to enter the millstone into it, in such a way as to leave sufficient space between the stone and the rods for taking pictures.
Some recognizable markers should be placed in different points of the millstone: these are for aligning and merging the two point clouds that will be generated (see below). A simple solution is to use small spheres of coloured modelling clay visible in the point clouds.



At this stage, you start with the typical workflow of the SfM. You take an appropriate number of pictures of the upper surface first; then do the same to the lower surface, turning upside down the millstone inside the frame.
The pictures can be processed by the software you want. I used Regard3D / OpenMVG for generating two point clouds (one for the upper surface and the other for the lower) and CloudCompare for editing/cleaning the point clouds and for performing their rectification (thanks to the cross marks on the frame), alignment and merging (thanks to the coloured markers on the stone). CloudCompare and MeshLab have also been used for generating meshes and for computing other parameters, among which the measurements.



The final result is a point cloud and a mesh of the millstone.


Using MeshLab you could also obtain the texture of the object, but for my aims it’s enough a 3D model (point clouds or mesh) from which I can get a plan, some cross sections and all the measures I need. Thanks to these data, I can detail or check my handmade drawing or do it from scratch.


In conclusion, the usage of an homemade wooden frame makes easier and more precise the data acquisition for the SfM and make faster and more complete the documentation of this kind of artefacts. The method described leaves room for improvements and developments; it could become a “standard” documentation technique for the ancient millstones and for other archaeological objects with analogous drawing issues.

Thank’s to Alessandro Bezzi (Arc-Team).

Denis Francisci




Saturday, 26 August 2017

Mapping high alpine lakes for archaeologic explorations

Hi all,
as you see we are writing few post in ATOR in the summer season, due to different field projects which take us away from home. Today I try to start again to dedicate some time to our research blog.
The topic of this post regard a solution we are currently using to help us in the archaeological exploration of high alpine lakes: the documentation of the bathymetry through a low cost sonar.
As you maybe know, since a couple of year we are working on underwater archaeology projects in the alpine lakes of our region (here an example). This kind of exploratory mission are difficult, due to the altitude of the site we have to investigate (almost always over 2000 meters asl), so that our divers have to acclimate themselves for one whole day, before starting the working. Also for this reason we started again to study archeorobotics and develop, together with our friends of the WitLab, an open hardware ROV called ArcheoROV (in order to help divers in exploratory mission).


The ArcheoROV (photo by WitLab)
 
This year we focused or research in find a cheap solution to map the bathymetry of the lakes, while WitLab went on working on the  Wi-Fi buoy which gives our ROV a long-range operability (respect the limitation of a simple control on shore). For this reason we tested a cheap sonar called Deeper, which normally is used as a fishfinder.
We started our test in the Lake Tovel, thanks to the hep of Prof. +Tiziano Camagna , who is leading the exploration project since many years. This lake is almost our playground to develop and test new solutions for underwater archaeology, since it is a difficult environment, but not extreme (like other high mountain lakes). We chose this location also because, on unlike other lakes, its bathimetry was documented by Edgardo Baldi in the 30s. We already digitized this map, processing a 3D model in GRASS GIS, so that we have some data to check our results with our small sonar (as you can see in the image below).

On the left the map drawn by Edgardo Baldi between 1937 and 1938; on the right the 3D derived map developed by Arc-Team in GRASS GIS

Some more details of the 3D map developed with GRASS GIS

To test the Deeper sonar, Porf. +Tiziano Camagna designed a small buoy which can be towed by a kayak. This solution stabilize the sonar (which remain always in the right position) and, at the same time, avoid its submersion (which causes the lost of the GPS signal).

The stabilization buoy developed by Prof. +Tiziano Camagna 

First positive results (image below) encouraged us to use this solution on a real mission, at the Monticello lake (almost 2600 meters asl), at Paradiso Pass (near Tonale Pass, Trentino, Italy).


A comparison between the digitized map of E. Baldi (on the left) and the map (work in progress) obtained with the Deeper sonar (on the right)

The expedition was joined also by our friends of the Team Nauticamare (Massimiliano Canossa and Nicola Boninsegna) and gave us the opportunity to accomplish a first mapping of the Lake Monticello, during the first day of acclimatization. This helped us very much during the archaeological underwater mission of the second day. As a result we have now a good 3D map of the bathymetry of the lake, which we will use also in the next expedition (September 2017). Her below is a short video (done with +QGIS plugin qgis2threejs), which shows the 3D model of the lake.




PS
I recorded some videotutorial related with the processing of these data. I will try to upload them ASAP in our channel.

Have a nice day!


Sunday, 30 July 2017

The Lady of K'anamarka and the relevance of computer graphics for tourism



For the eighth time, I traveled to Peru, as I have always been very well treated and enjoyed good moments throughout the social spectrum. The big difference from this episode in relation to the other seven is perhaps the fact that I realized how graphic computing might be relevant to apparently disconnected areas of their initial claims, such as tourism. In the face of this I will share with you my reflections and history of how things unfolded in that beautiful Andean country.

Stages of facial reconstruction of the Lady of K'anamarka. This woman was a victim of a trepanation surgery on her modified skull (artificially deformed skull, "ET skull").

Something similar had happened in previous episodes, it is enough to mention the facial reconstruction of Gufán (the Brazilian of 2000 years) that took in one day the number of visitors of a month to the Museum Paranaense. The presentation of the bust of St. Valentine in St. Valentine of the South city (Brazil), which crowded a 1200-seat church in a town of just over 2000 inhabitants. In addition to these two facts, I can not fail to mention the reconstruction of the men of Lagoa Santa, a feat propagated in the national and international media, which, of course, drew attention to the archaeological site of the city, as well as considerably increased the number of visitors to the Museum of the Lapinha.
Views of the skull of the Lady of K'anamarka

The Lady of K'anamarka


Returning to Peru, the first project presented there was the face of the Lady of K'anamarka, whose remains were found by the archaeologist Marco del Pezo in 2004. I was invited to participate in this project by physical anthropologists Dr. Oliver Medina and Dr. Elva Torres. The work was tranquil, since Dr. Medina is a specialist in photogrammetry and passed me the richly digitized skull in 3D. From there I built the face and designed its presentation with the help of the Inca Garcilaso de la Vega University, of which I am part of the team and with whom I developed a series of projects in that country.


Some interesting events guided the presentation of the face. Initially the revelation of the face was scheduled to be made in a room with 120 seats. On the first day of the seminar where we were introduced, the audience showed up in weight and many people had to watch the event standing up. This forced the organizers to change rooms, the other day the face presentation happened in a space for 400 seats! The press appeared in weight and the news of the reconstruction became headline in the main newspapers and TVs of Peru.


The other day I was invited to visit the archaeological site of K'anamarka, where they discovered the skull. A four-hour trip from Cusco, where we met. Beyond the distance we would climb a little higher, towards 4000 meters of altitude! I feared for my breath and my stomach, but fortunately for all indications I have excellent resistance to altitude, because I felt nothing different, except for the air that was not as abundant as in the lower parts.


Arriving at the site I was treated almost like a head of state, at least it was what I felt. After a four-hour drive and in that place of clean, wonderful visual air, I could see that a delegation was already waiting for us at the entrance to the old city. As I descended, I was approached by reporters, guides and representatives of the Municipality of Espinar, who gave me some gifts, among them showy local clothes, which softened the cold and warmed the heart with so much human warmth.


Each step we took was accompanied by a series of people, all properly filmed and photographed.


After visiting all points of the site, we were treated to a delicious lunch, with delicacies from the local cuisine. The soul was fed, but the body was not yet, we did not waste time and devoured that delicacy, closing the bill with a local drink that aided in digestion.


The other day we had the great joy of seeing the enormous repercussion of the work in newsstands and national TVs. More than that, TV Latina, one of the main ones of Peru made a special matter from the facial reconstruction. The material they composed (thanks immensely to the reporter Jesus Noriega) addressed our work and gave a tremendous emphasis to tourism of the region, while showing the natural beauties and qualities of its warm people. In total there were 17 minutes of video transmitted to the whole country!


The statue of the Lord of Sipán


The facial reconstruction of the Lord of Sipán was a work begun in 2016, also in partnership with the Inca Garcilaso de la Vega University, the EBRAFOL and the Royal Tombs of Sipán Museum. That year we unveiled the face digitally and the event became news worldwide, being propagated in 27 languages!


The closing of this project took place in great style, with the printing of the bust by the CTI Renato Archer and the painting of the piece by the skilled hands of the artist Mari Bueno, award-winning Brazilian painter, whose works were the theme of exhibitions in Brazil and in other countries.


Unlike the 2016 event that took place in Lima, this was held in Lambayeque, more precisely at the archaeological site where they found the body of the Lord of Sipan. In addition to the presence of Dr. Walter Alva, the archaeologist who discovered it, the Peruvian Minister of Culture, Salvador del Solar, a former actor very dear to the population, whose management has high approval ratings, Cultural development.



I was very happy because my presence and that of Dr. Eduardo Ugaz Burga, dean of Garcilaso University was informed during the ceremony and things did not stop there. Soon after the revelation of the bust, me and the university were awarded an honor offered by the Regional Government of Lambayeque.

Like K'anamarka, the other day the newspapers and TVs stamped the presentation of the statue of the Lord of Sipan.

Conclusion


I'm kind of tired of talking about it, but research work is almost always an activity that happens away from the knowledge of the general public and the spotlight of the press. However, when this work is recognized the feeling of gratitude goes beyond the limits of the body and the smile. I am honored to promote the tourism of a region and, in a way, to make known the places where I present the faces. The joy of the conquest is only tainted, when distance myself from the great friends that I am getting on these trips.


Monday, 3 July 2017

Faces and FACCE much more than two exhibitions


I met the staff of Arc-Team in 2011, it is an archaeological research company that works mostly with free software. My first contact was through a post about photogrammetry, that is, the technique of scanning a 3D object using photographs.

Although not so distant, that year I knew little about 3D scanning, 3D printing, reconstruction of CT scans and even, almost nothing about forensic facial reconstruction.


I started almost parallel studies of photogrammetry and facial reconstruction and, thanks to the good relationship I developed with the Arc-Team staff, I was able to put these two fields together in a very nice project, the reconstruction of the face of the Taung Child in 2012.


This project was a partnership between Arc-Tem, myself, the University of Padova and the NGO Antrocom. We barely imagined future ramifications of all this.

Encouraged by the technical repercussion of this first attempt, I began to outline a new project, this time, having the archaeologist Moacir Elias Santos in the team. Together with him and supported by the Egyptian Museum and Rosicrucian, we organized the first exhibition of facial reconstruction applied to human evolution, called Faces of Evolution, which was inaugurated in 2013.

Inspired by this exhibition, the Arc-Team and the Museum of Anthropology of the University of Padova started a new venture, an exhibition focused on the human face and our ancestors of evolution. Your name: FACCE - i molti volti della storia umana.


While the Brazilian exhibition was composed of 11 posters and replicas of reconstructions, the Italian exhibition would have 27 facial reconstructions, 5 of them historical and 22 evolutionary, as well as other environments that would touch the various facets of a face, around the world, At different times and in different philosophical approaches.

Still at the end of 2013 I was in charge of reconstructing the face of St. Anthony of Padua. This reconstruction marked the whole year of 2014, since the work was news on the main TV channels in Brazil and Italy, even extrapolating the limits of those being published in 18 languages!

The FACCE show was finally inaugurated in early 2015, and in the face of its success, it was extended until the end of that year.

In 2016, using the same material from this show, with the addition of a new hominid, Homo naledi, Facce da Evoluzione exhibition was auctioned in Genoa.

Now, in 2017, I decided to make a survey, about the direct and indirect repercussion of this work, which began in 2012 and little by little took our technique to the four corners of the world. For this, I created a clipping with all the reports involving the reconstructions I worked with, as well as the languages in which they were published. Why languages? Because it would be simply impossible to organize all this based on the countries or media. By focusing on the languages, we attest to how far the work has come.


The work that had more occurrences in different languages was the reconstruction of Paranthropus boisei, carried out in 2013, 21 in total. Then, the already mentioned face of Santo Antonio and Homo erectus, both with 18 occurrences. In the table above we can see the other works and their numbers.

The total was 43 different languages, which together total 5.26 billion speakers, meaning a considerable part of the world can read the news stories with our work!


The languages with the highest occurrence were English and Portuguese, which appeared in 15 citations of the 15 selected terms. A little behind appears the Spanish (14), followed Italian (14), Russian (11) and Turkish (9).

The most impressive, was to attest that important sites such as the BBC, Discovery Channel, History Channel, National Geographic, Scientific American, Science News, Wired, RAI, Globo TV, Encyclopedia Britannica, The Washington Post, Science News and Homo naledi  and Australopithecus sediba own discoverer, Dr. Lee Berger, shared stories by way of illustration of reconstructions made for the exhibits covered in this post.

The Arc-Team group always has the mantra to share the results of the work, so all reconstruction images were donated to the Wikimedia Commons under free license. That was one of the main factors for all this repercussion.

The work does not stop here, there is much to come and many fruits will emerge as a result of research in the field of facial reconstruction of modern men and ancestors of human evolution. The knowledge that I gained in the facial approach of these fossils has helped me even in one of the areas that I work professionally, the planning of facial surgeries, as well as accident monitoring tools. It seems unlikely that such separate areas have points in common, but they are more numerous than you might think.

As a member of this fantastic team I can only say that we have achieved our main goal, which was to make our work accessible and to make it free so that as many people could use them. May the next come!

Thursday, 27 April 2017

ArcheOS Hypatia and Archeorobotics: ROS for 3D dynamic documentations

Dynamic 3D documentation is a technique we are using more and more in professional archaeology. It can be useful to map in a very fast way any kind of earth-moving work during negative archaeological controls (from wide open area to small trenches, like we did here) or to record in real-time archaeological evidences and layers during a normal excavation (like the video below).




Using this methodology during an ordinary project allows us to perform the segmentation of the 3D model directly on the field (within the software Cloud Compare), dividing each layer of palimpsestic documentation (we spoke about this problem during our presentation at the CHNT conference 2016). This solution avoid long post-processing operations, and it is ideal to spare time and money in low-budget archaeological investigations. For this reason we are evaluating the possibility to insert ROS (Robot Operating System) in ArcheOS Hypatia.
I hope to give you soon good news about the release of the next version of ArcheOS. In the meantime stay tuned to follow our research in testing new Open Source and Free Software. Like always, if you want to help in the development, just contact us in one of our channels: FaceBook, YouTube or Blogger
Have a nice day! 

Monday, 24 April 2017

ArcheOS Hypatia Virtual Globe: Cesium

Hi all,
I am starting here a series of short post to show some of the features of the main software selected for ArcheOS Hypatia, trying to explain the reasons of these choices. The first category I'll deal with is the one of Virtual Globes. Among the many available options of FLOSS, one of the applications which meets the needs of archaeology is certainly Cesium. This short video shows its capability of import geolocalized 3D complex models, which is a very important possibility for archaeologist. In this example I imported in Cesium the 3D model (done with Structure from Motion) of a a small boat which lies on the bottom of an alpine lake (more info in this post).


Soon I'll post other short videos to show other features of Cesium. Have a nice evening!
BlogItalia - La directory italiana dei blog Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.