Robots create hands-on feeling for remote mine operators

In the old days, miners would take a canary down to work with them in the mine shaft. If the canary died, the miners knew that they’d soon be next if they didn’t make a run for it. Canaries were a good indicator of pending disaster.

But canaries can’t do algorithms

In the mine shafts of the future, it will be robots not canaries that act as sentinels for safety. They’ll also be programmed to support or carry out other aspects of mining operations, while human operators stay safely positioned in remote stations above ground. But in the absence of any warm blooded beings below, how will remote operators be able to get a hands-on feeling of what’s going on? In order for mining operations to remain robust, efficient and safe, these remote operators can’t just be observers. They must partake in the operations, and with different senses in play.

A sense of sight can be achieved with various types of remote camera visualizations, although it’s still not easy for operators in the absence of 3D or stereoscopic view. But what about the other senses? Robots need to give feedback to the extent that remote operators feel just like being there on the spot with the remote machine.

Putting all the senses in play

An operator using a subterranean robot to drill holes in rock face may be able to experience a sense of touch, for example, through haptic feedback on the remote controllers. That means being able to feel how solid the rock is and adjust speed or strength accordingly. If the rock were to get too hard for the drilling equipment, it could create all kinds of unwanted safety repercussions and the need for expensive repairs.

shutterstock_132947174A sense of hearing as well as a sense of smell would also be useful, for example, to analyze machine motor sounds and underground explosions or events where gas leaks or burning are indicators of pending disaster. In the former case, auditory feedback is rather easy to implement with microphones, speakers and deliberate sound design. But in the latter case, instead of direct olfactory, or sense of smell, feedback, applicable sensors need to be able to detect suspicious alterations in air quality and alarm operators in other ways.

We recently visited a mining and technology conference in Toronto where many of the presenters and participants were using buzzwords like IoT, Industry 4.0, augmented and virtual reality (AR/VR), user experience and autonomous systems. However, during the breaks, most conceded that the mining industry still has some way to go before these tech trends could be applied to deliver value.

How can VTT help?

In a rather different kind of environment to mines, VTT, together with Tampere University, has developed a Remote Operation and Virtual Reality Center (ROViR) in Tampere, Finland to support remote operation and maintenance simulations for one of the world’s most challenging energy projects, based in the South of France, called ITER – in Latin, meaning ‘the way’.

As with mineshafts, no warm-blooded beings will be able to enter the ITER facility once it’s up and running. That means all maintenance will have to be carried out by smart robots, equipped with sensors for dealing with every possible scenario.

Finding the way with ITER

So far at ROViR, VTT has achieved high success in simulating ITER remote maintenance operations, many of which can be applied to other sectors. For example, VTT has honed the use of a transport robot to move a ten-tonne reactor cassette along a desired route with an accuracy of plus or minus 1 mm.

As well as supporting ITER, one of the aims of our design work with ROViR has been to find new ways for remote operators to achieve a sense of control. And many of these solutions can be directly applied to mining. For example, using VR/AR to help operators better understand spatial dimensions, such as distances among elements; applying ecological interface design for safety-critical control room monitoring user interfaces; or using AR video feeds to highlight crucial objects in remote work areas.

Our InnoLeap concept design approach has been developed to push radical new concepts into the industrial workplace. Backed by the experience gained from this concept design work with ROViR center, we’re now well positioned to provide numerous applications for other industries, especially the mining sector. That means collaborating with mining companies to go deep into the remote operators’ user experience. Using our task-analysis methods, we can explore what remote mining operators will need to do and know in order to be successful in their work tasks, both in abnormal and normal situations.

Canary or no canary, remote operators in the mines of the future won’t succeed by just winging it.

For more information, please visit: www.vtt.fi/innoleap/

hannu_karvonen_kuva
Hannu Karvonen
Research Scientist, VTT
hannu.karvonen(a)vtt.fi

 

 

OLYMPUS DIGITAL CAMERA
Mikael Wahlström
Senior Scientist, VTT
mikael.wahlstrom(a)vtt.fi

 

 

What will your industry look like in 2030?

Industry is reinventing itself around digitalization. New technology buzzwords like user experience, augmented and virtual reality, automation, autonomous systems, and Industry 4.0 are all becoming mainstream. But every industry has the same problem: how should the new technologies be applied to deliver value in a disruptive market?

Change is coming

When we go out into the field, customer discussions turn quickly to the biggest fear they face. Fear of missing the boat or losing their competitive advantage. Change is coming but they don’t know what that change looks like, only that someone, somewhere might be about to steal their lunch.

Sun Tzu in the Art of Warfare said “To know your enemy, you must become your enemy.” We don’t think he had technology wars in mind. But what if we apply his thinking anyway to today’s competitive markets? To combat change you have to become the change. And if you don’t know what change looks like, the best way to predict it is to create it yourself – in collaboration with the right partners. InnoLeap_text_CMYK

At VTT, we developed InnoLeap to serve partner companies who have taken precisely this kind of proactive stance towards changing conditions. So instead of waiting around to see what change your industry will deliver up next, why not take the lead and become that change. But how?

Disrupt your own domain

The best place to start is to visualize change by developing new and future-oriented solution concepts. We can work with you to deliver them in ways that are engaging for your customers, the media, and other broader stakeholder groups.

Some of our jointly developed concept solutions have already tested well, with a surprising degree of success. Rolls-Royce Marine piloted InnoLeap and our collaboration journey led us to a whole new vision for unmanned ship operations in 2030. With this shared project, not only did we go together some way towards disrupting the marine domain, but we also caused a shift in thinking towards renewal of the entire industry – in this case, through visualizations of the operation of autonomous ships.

InnoLeap, if we unpack it, is basically a collection of concept design principles to push radical new concepts into the industrial workplace. Backed by our experience and solid research methods, we can go deep into the users’ world and experience while at the same time wowing your stakeholders with previously unimagined designs: with task-analysis methods, we explore what the users need to do and know in order to be successful in their work tasks, both in abnormal and normal situations. Together with the client, we can then combine this understanding with knowledge on technology trends for creating new solutions of the future.

Be your industry’s new thought leader​

With Rolls-Royce Marine, VTT was able to work with the media in a third-party capacity, precluding the need for our client to self promote. And in the first three weeks, our concept releases, pictures and video had garnered 250 separate news articles and 40,000 YouTube hits.

Rolls-Royce Marine definitely raised its profile, repositioning itself in the industry as the new thought leader, and the company is now working to influence stakeholders, including maritime lawmakers, to help drive these new concepts forward. Which means, when it comes to radical industry change, Rolls-Royce Marine is definitely not missing the boat.

For more information, please visit: www.vtt.fi/innoleap/

SEE: Rolls-Royce shore control centre concept video
OLYMPUS DIGITAL CAMERA

Mikael Wahlström
Senior Scientist, VTT
mikael.wahlstrom(a)vtt.fi

 

hannu_karvonen_kuva

 

Hannu Karvonen
Research Scientist, VTT
hannu.karvonen(a)vtt.fi

 

Find out what your users want, but don’t give it to them!

User experience can drive innovation, but if you’re looking for epoch-making innovation, don’t look only at what the users want.

In our user research work, we talk a lot to professional users. We interview them on what they do with a view to wrapping our findings into product and service concepts. The irony of our work is that even though user experience data is invaluable, if we really want to come up with a big new domain disruptor concept, we’re better off not going along what the users wish for.

Hands on learning in the Deep South

We were a few years back on the East Coast of the US doing fieldwork for Konecranes. The plan was to interview local crane operators, as part of a core-task analysis to discover the general demands of their work. These findings would eventually fold into a prototype system for a remote operator station, delivered through our InnoLeap approach.

Down at the wharf side, they can be a bunch of tough guys, and not keen on being interviewed by Finnish scientists. After a few false starts, we decided to bin the interview plan and settle for small talk. Although not a Finnish specialty, once we’d introduced the topic of last night’s ball game, we quickly found ourselves in full conversation and sharing mode.

In terms of developing an empathetic understanding of the users, we were well on our way. These crane operators knew exactly what they were doing and why. After years on the job, they had an intuitive nose for their operations making them able to sniff out potential problems and analyze the safety of their operations.

shutterstock_569308957

However, once we started asking them for ideas on how they could do things more easily, more safely and more efficiently, for example through automated operations, they were generally stumped. In most cases, we find that users are too closely wedded to their current systems and practices to be intuitively able to even conceive of a radical new approach. On top of that, users are not necessarily up to speed with the whole gambit of technical possibilities or trends that the future has on offer.

Intelligent towing for ghost ships

As another case in point, the InnoLeap team worked with Rolls-Royce Marine, to develop new concepts for future ship bridges. One important user study finding was the need for tugboat operators escorting large ships to constantly anticipate the movement of the bigger ships being towed, especially in turbulent seas. Massive cargo ships turn slowly so tugs need to assist in an anticipatory way before the big vessel gets into trouble.

shutterstock_591360698

Since these ships of the future may be autonomous, or ghost ships, as some of the users called them, the design goal had to be remote but highly intelligent. Taking into account the user findings and design goals, we came up with, for example, one concept solution called Intelligent Towing, which involved a direct data transfer from the big ship to the digital window head-up display of the tugboat. Information included the speed, turning rate, and distance between the two vessels, as well as the rate of strain directed on the towline.

How to address the irony of user research and radical innovation

The developed concepts and their visualizations scored a very high wow factor from the industry as well as technology pundits and even mainstream media. The secret to success from our side was to involve the user experience perspective only once we’d achieved what we call the fuzzy front end of design. This is the time when we’re free to come up with all kinds of initial concepts, even bordering on the absurd. Later we’ll have a chance to temper these ideas against the true user experience with a view to one day turning ideas into real marketable solutions.

This was also our intention in the fieldwork with Konecranes. At the end of our visit with the wharfies, we gave a hearty thank you and talked of a possible return visit to do some evaluations of our designs – and maybe even catch another ball game.

For more information, please visit: www.vtt.fi/innoleap/

hannu_karvonen_kuva
Hannu Karvonen
Research Scientist, VTT
hannu.karvonen(a)vtt.fi

 

 

OLYMPUS DIGITAL CAMERA
Mikael Wahlström
Senior Scientist, VTT
mikael.wahlstrom(a)vtt.fi