How did the Ukrainian tech specialists invent the unique solution for the virtual reconstruction of historical objects?
Ukrainian tech giant SoftServe presented the first augmented reality app that recreates a 3D model of a wooden medieval Ukrainian castle Tustan located in the Lviv region, Ukraine – in its former glory in the rocky hillside. The uniqueness of the project is in the detailed and precise reproduction of the wooden structure of the Middle Ages castle.
To begin developing this app, SoftServe needed to address several challenges, as all that remains of the fortress today are the sites where wooden blocks were historical. The area is not similar to other traditional historical sites. In addition, the rocks had crumbled over time, which changed the original outline. SoftServe had to make sure the rocks in the developed 3D model matched those in reality.
SoftServe’s R&D team of 3 tech experts worked on the project within a year. Key steps during the development of the project included:
- Physical measurements of the location, examining the existing materials and plans. The use of drones aided in reaching the less accessible areas.
- Photogrammetric reconstruction for digital graphic models of the rocks to within millimeters of accuracy.
- Ensuring that the rocks depicted in the photographs coincided with the rocks in the real environment, and the rocks on the developed 3D model that had to match the actual landscape, taking into account the depth and clipping.
- To pick up several parameters (focal length and physical size of the sensor, distortion of the lens, zoom, and rotation) quickly and accurately, SoftServe used several algorithms for mathematical modeling of reconstruction.
- Technologies: BLAM plugin for Blender, OpenCV, LAStools, photogrammetry techniques, MeshLab, SFM (structure from motion), MVS (multi-view stereo) algorithms, 3D transformation models.
Presented in 2020, today the Tustan AR mobile app is live in real-time on the location and is available in any weather conditions. Thanks to the project the ancient Tustan Castle can be viewed remotely by downloading a free app in the App Store or the Play Market.
“Building the Tustan AR app was a unique challenge with new levels of complexity. Working closely with Vasyl Rozhko who founded the historical landmark and the ‘Tu Stan!’ festival, and who is the initiator of the project, we shared his passion to help bring this historic castle back to life. The solution we developed can be applied to the virtual reconstruction of other historical sites in Ukraine and all over the world. We are ready to share our best practices for the preservation of historical heritage.” – Semen Frish, AR/VR Engineer at SoftServe.