New mixed-reality holographic projector for construction sites

Construction professionals can now ‘see’ a building in place and in rich detail before a single brick has been laid

Soldiers have a saying, ‘No battle plan survives the first contact with the enemy.’

In a less dangerous way, the construction industry also has to deal with the variances between the ideal plan on paper – in the form of an architect’s drawings – and the reality on the ground. The architect’s design is a vision, but the job of realising this vision is the builder’s.

For centuries, the main way in which architects have communicated their vision to builders is in the form of a plan, or drawings: a 2D, abstract representation of 3D reality. Even today, when digital systems have replaced paper-based methods in so many domains, builders are still tied to computer renderings of an architect’s plan – drawings which are still 2D when viewed on a display screen.

Mentally projecting a flat, abstract representation on to 3D reality – in other words, imagining how the finished structure will look on-site before a single brick has been laid – calls for skill, judgement and interpretation on the part of building professionals. Inevitably, this creates scope for error, miscalculation and miscommunication between members of a construction project’s team.

Fig. 1: the EBMR mixed-reality helmet in use on a construction site

Now a completely new way for builders to see an architect’s design on-site has been developed by a joint venture between Dura Vermeer, a renowned family owned Dutch construction company specializes in the development, engineering, construction and maintenance of (commercial) real estate as well as infrastructure, and Recreate, a developer of augmented reality software. Called the EBMR (Engineering Building Mixed Reality) helmet, it provides for the first time a 3D, real-time holographic rendering of an architect’s plan overlaid on the user’s view of the building site. Standing on an empty building site, the user can see the structure, correctly positioned, as though it had already been built (see Figure 1). As the user moves, their view of the structure moves. The user no longer has to imagine how the building will look, based on an understanding of the drawings: the EBMR helmet shows the user exactly how it will look, without ever having seen the drawings.

The development of the EBMR helmet is a remarkable technical achievement – and it depends on an advanced combination of cutting-edge hardware and software technologies including the Xsens MTi-300.

A different kind of hard hat

Donning a helmet is second nature to building professionals: safety regulations require them to wear a hard hat on site. So Dura Vermeer’s idea of creating a hard hat for builders to show them what they were to build was very much in keeping with their role as one of The Netherlands’ leading building companies.

Dura Vermeer brought to the EBMR project some 160 years of experience in the construction industry. It knows intimately the difficulty of translating an architect’s drawing into a detailed, step-by-step construction plan, and of co-ordinating the implementation of the construction plan across large teams sometimes consisting of hundreds of builders. Together with the knowledge of the development team of Recreate, who have been working with technologies such as Virtual Reality, Augmented and Mixed Reality for years, software has been developed to visualize 3D models with the Microsoft HoloLens.

The master document which guides the management of construction projects such as those undertaken by Dura Vermeer is the architect’s blueprint. But the blueprint is not the same thing as an instruction to a builder. For instance, an architect’s drawing might call for a wall to be built. The builder needs to know exactly where the hole for the foundations needs to be dug. A drawing on paper or on a computer cannot point to a spot in the mud and say, ‘Dig your hole here.’

Fig. 2: the Microsoft HoloLens projects an architectural design on to the user’s view of the building site

The EBMR helmet can. It provides a real-world projection of the drawing, so that a builder can walk around a site and ‘see’ exactly where each element of the structure will go (see Figure 2). Using ultra-accurate satellite positioning data, the helmet knows exactly where the user is on the surface of the earth. From the drawings, it knows exactly where on the surface of the earth the structure is to be built. It can then ‘build’ the structure in virtual reality and present it to the user, accurately positioned and oriented. And because it is presented holographically, the structure is overlaid on the user’s view of the site in front of them – a ‘mixed-reality’ display.

The EBMR helmet has already been used in pilot projects, such as the construction by Dura Vermeer of a bridge in The Netherlands, and has provided three valuable benefits in them:

• Time saving – the helmet enables construction and site managers to make faster and better decisions about the sequencing of tasks, about the allocation of assets such as construction equipment, and about the procurement and storage of materials. This leads to reduced wastage of time, assets and materials.
• Cost saving – the EBMR helmet is so precise that it can be used to make on-site measurements that would previously have required the services of a professional surveyor. Now, such measurements can be taken by ordinary members of the construction crew. Because the use of the helmet also reduces the potential for mistakes, there is less requirement for costly and time-consuming rework.
• Quality improvement – the EBMR helmet eliminates the requirement for judgement and interpretation of the architect’s drawings. When builders can look at the site and see what needs to be built in the location where it needs to be built, their work tends to more exactly match the architect’s intention with zero defects or variances.

A combination of best-in-class technologies

To provide the capabilities of the EBMR helmet, Dura Vermeer and Recreate required four main hardware elements (see Figure 3):

• A holographic projector - this is a Microsoft HoloLens device mounted at the front of the helmet. The Hololens is the world’s most widely used holographic device for professional applications.
• A positioning system - this is a Global Navigation Satellite System (GNSS) receiver mounted at the top of the helmet. GNSS receivers are small semiconductor-based modules, and are commonly used in vehicles and mobile phones to support navigation and mapping applications. The EBMR helmet uses a precise GNSS receiver to provide accurate positioning co-ordinates.
• A battery power supply - this needs to provide sufficient capacity to power the device for a whole day between charges, as charging on-site might not always be possible.
• An Xsens MTi-300 - a type of motion sensor module which provides orientation and heading information. Because of the Xsens motion tracker (MTi-300), the helmet knows in which direction the user is facing, and the position of the head. In navigation terminology, this positioning information is known as pitch, roll and yaw. The heading information is a compass reading relative to magnetic North.

Fig. 3: The EBMR helmet is in a standard hard hat format and is nowadays comfortable enough to be worn for long periods. The picture shown is the 1^st version of the helmet.

To avoid the risk of magnetic interference between the other electronics systems and the IMU, which contains an extremely sensitive magnetometer, or magnetic compass, the IMU is mounted at the rear of the helmet.

Xsens is a specialist in the provision of wearable IMUs. It is the world’s leading provider of motion-tracking bodysuits, which are worn by actors in the production of animated films for Hollywood studios, and by the automotive industry’s crash test dummies. Xsens IMUs are popular in these applications because they combine compact dimensions and ultra-low power consumption with high accuracy. Xsens also provides system developers with a comprehensive development tool suite and configuration tools to enable rapid optimisation of the characteristics of motion-tracking and orientation functions in any application.

The other essential element of the EBMR system is provided by Recreate: the software which converts an architect’s computerised drawings into the holographic image projected by the HoloLens. The software fuses the information in the drawings with real-time positioning and orientation information from the IMU and the GNSS receiver to project a holographic image which exactly represents the viewer’s perspective of the virtual building in both the horizontal and vertical planes.

EBMR: a revolution in the techniques of construction planning and management

The performance of the EBMR has been proven in detailed field trials: its visualisation of a building design is accurate enough to be used in place of professional surveying equipment. Comfortable enough to be worn for long periods, it has been enthusiastically welcomed by construction crews. It is a revolutionary development, replacing the centuries-old 2D representation of building designs and substituting a 3D projection which merges seamlessly in mixed reality with the user’s real-time view of the construction site.

The EBMR helmet is already helping Dura Vermeer to accelerate building projects, reduce costs and improve the quality of construction in large and complex civil engineering and construction projects.

The EBMR helmet: artist’s impressions