Robot Programming and Simulation using Virtual Reality Techniques

Márcio S. Pinho¹, Antonio A. A. Coelho¹, Heitor A. O. and Silva¹,
Renato M. of Silva¹, Fernando Shirmbeck², Marcelo Cohen¹,
Fernando H. Cardoso², Marlise S. of Santos², Ângela Mazzorani¹

¹Pontifícia Universidade Católica do Rio Grande do Sul
School of computer Science/School of Engineering
Av. Ipiranga, 6681–Porto Alegre–RS–91.360-080–Brasil
[email protected]

² National Service for Industrial Education (SENAI-RS)

Abstract: This work presents the design and development of a robot movement system using virtual reality devices, as a glove and a three-dimensional position tracker. By using these devices the user can make, with his hand, the movements he wishes and the robot will repeat them. The system captures the user’s movements and translates them to robot commands. The system implements also a three-dimensional environment simulation environment where many programming tests can be made.

Keywords: Virtual reality, robotics

Project partially supported by FAPERGS

1. The Problem Context

This work is in the context of Programming Environments for Articulated Arm Robots, very common in Technical and Engineering Schools.

We began this work in 1997 with the goal of creating alternatives to the robots' programming process. We worked mainly at the laboratories of National Service for Industrial Education(SENAI-RS) and of the Polytechnic School at PUCRS.

In these environments students and researchers are trained in robots' programming. In the development of this task, there are two basic restrictions:

• the difficulty to programming complex movements;

• the little availability of the equipment (robots) to accomplish exhausting tests.

An articulate arm robot is usually handled in two basic ways:

• through controllers with buttons or some other control device (i.e. a keyboard);

• through commands in a programming language.

In the first case, a great ability is demanded from the operator because he needs to map from the controller’s movements to the robot's movements to ensure the equipment executes the desired action. This necessity exists because the movements accomplished in these devices(keyboards, or control buttons), are not physically identical to the movements accomplished by the robot. For instance, in some systems, to elevate the robot’s grip, a control handle should be moved to the left or to the right.

In the case of using a programming language, in the first place, it is necessary to do several tests to check if the program executes the desired action. Besides, the indispensable programmer's capacity in solving problems with algorithms, a task which is not always trivial in the existing context.

Thinking about these problems we decided to use virtual reality techniques in a way that the operator could use his own arm to move the robot. With this, the use of the robot becomes easier and safer, since the robot follows the movement of the operator’s arm.

Besides, we decided to create a three-dimensional visual representation of the robot(a 3D simulator). With this software, the operator can test his ideas as many times as he needs without having the need to have the robot present in fact. Plugging virtual reality glasses(a HMD- a Head Mounted Display) to the simulator it allows also the operator to have the sensation of being present in the same environment as the robot is.

1.1 The Robot

The robot used in this project is the SCORBOT-ER VII (figure 1). This robot, manufactured by the israeli company Eshed Robotec Inc.[ESHED 1992], consists of a mechanical arm composed by 5(five) articulations (base, shoulder, elbow and wrist, this last one composed by 2 articulations) and 1 grip (with 2 stages: open and closed), according to figure 2.

Figure 1 - SCORBOT-ER VII Robot

Figure 2 – The SCORBOT-ER VII articulations

The SCORBOT-ER VII uses as programming language the ACL - Advanced Control Language [ACL 1992], specific for the robot controller.

The connection between the computer and the controller is done via serial port. To send commands to the controller, the user has two options: to use the ATS (Advanced Terminal Software) program or to create his own program that sends the ACL commands through the serial interface.

2. The developed solution

This work developed a teleoperation system for a mechanical arm SCORBOT-ER-VII, through a virtual reality environment.

In this environment the user can use his arm movements to move the mechanical arm. This provides a very easy way to create complex movements, sometimes, not easy to program using traditional methods. Besides, the system created allows the three-dimensional visualization of the robot, during the operation work and it still provides a programming and simulation tool, that allows the execution of many tests without having to be in the presence of the real robot.

Through the system use, the robot's operator can control it using a virtual reality glove and a three-dimensional position tracker(figure 3). The user, through the glove and the 3D tracker, can move his arm, accomplishing the movement to be made by the robot. The system, then, captures the positions of the user's hand in space and it converts them, to commands in ACL programming language.

Figure 3 – The System’s Architecture

The robot's movements are sent to the controller unit through serial communication routines specially created for this system and compatible with the robot's communication protocol.

2.1 The virtual reality devices used by the System

During the project development a position tracker, a virtual reality HMD[PINHO 1996, PINHO 1997, CROSS-NEIRA 1998] and a data-glove were used to input the data.

The position of the user's hand is traced through an ISOTRACK II magnetic sensor from POLHEMUS Inc.[POLHEMUS 1999].

The tracker operation range is one meter, in each one of the three directions, starting from its base unit.

The HMD is a Virtual I-O Glasses, from the company I-O Display System Inc[I-glasses 1999].

The glove used in this project is a 5th Glove, manufactured by the General Reality Company [GENERAL 1998] (figure 4). The 5th Glove has an optic fiber sensors flexion to inform the movement of each finger. In the project, this device is used to capture the opening and closing movements of the user's hand, as well as the movements of the user's wrist. The data of the glove obtained are used to accomplish the necessary calculations to move only the grip of the mechanical arm. To use to 5th glove, we built a communication program capable to interact with it through its own communication protocol.

Figure 4 – The 5^th Glove

3. The Robot’s Movement

The robot’s movement is accomplished by the position calculation of the operator’s hand in space(from an initial reference), and by the conversion of these coordinates to the robot commands control.

The conversion from the user's hand positions to the positions where the robot should be moved is made considering the operation limits of the devices used (tracker and glove) and the limits of the robot's operation, configured during the system initialization.

After the calculation of the robot's new position, this is sent to it via serial interface using ACL commands. In figure 5 there is a small ACL program that moves the robot along the X axis.

setpvc 1 x 3807
setpvc 1 y -1300
setpvc 1 z 3600
move 1
setpvc 1 x 3400
setpvc 1 y -1300
setpvc 1 z 3600
move 1

Figure 5 – An Example of an ACL Program

3.1 Smoothing the path

Due to the high-speed updating rate of the position tracker, that can generate up to 60 readings per second, it is necessary to filter the data coming from it. This done not to overload the robot controller, that operates in a very inferior speed compared with the tracker rate. To do this, two premises were taken as base:

• several identical(or too close) points supplied by the tracker shows the user maintaing his hand stopped for some moments, not being necessary to send position commands to the robot;

• collinear points do not need to be sent in their totality. Some of them could be eliminated from the movement, without altering its trajectory(figure 6).

Figure 6–The Removal of Redundant Points

Using this technique it was possible to eliminate a large set of commands with no problem for the movement.

3.2 The paths storage

Another way to use the system, besides that one described previously (that forces the user to move his arm for each movement that the robot should accomplish), is to save the paths. This technique consists in saving the movements to be used later by the robot. This way, the user accomplishes the desired movement once and the robot can repeat it whenever it is necessary.

4. The Robot Simulator

After obtaining the robot's control using the virtual reality devices, we decided to create a three-dimensional environment where the robot could be simulated and the ACL programs could be tested, without necessarily having the robot in fact.

In this simulator the three-dimensional images from the robot’s arm are generated. These images(figure 7) were created using the OPENGL graphics library[WOO 1997, WOO 1998]. In the program, there is also the possibility to program the robot in the traditional method, using the language ACL.

Figure 7 – The robot's three-dimensional graphic representation

Trying to make the simulator the more similar to the real robot's environment, identical user-interfaces were used. This has made it possible for the simulator user's migration to the real world in a fast and easy way.

The generated three-dimensional images could be displayed in a monitor and/or in a HMD. To still give to the user a better immersion sensation[PAUSH 1998] in the robot's environment, a position tracker can be attached to the glasses. With this, the user can move around the robot as if this exists in fact.

5. Conclusion

The use of virtual reality devices in the robots' control is an alternative that has proved interesting to be used to improve the integration between man and machine. With the use of those devices the robots' operation is facilitated, and at the same time it decreases the risk of accidents in the work place.

The simulator, in its turn, is a very interesting alternative, especially in places where the robot is not available. With it, it is possible to accomplish as many tests as necessary, without the cost, nor the risks of manipulating a real robot.

6. References

[ACL 1992] ACL - Linguagem de controle avançada (guia de referência em português). Tel Aviv, Israel: Eshed Robotec Ltd., 1992.

[CRUZ-NEIRA 1998] Cruz-Neira, C. "Applied Virtual Reality". Siggraph’98. Course notes, 1998.

[ESHED 1992]  Eshed Robotec Limited, ACL - Linguagem de Controle Avançada - Guia de Referência e ATS - Terminal Avançado de Software, 1992.

[GENERAL 1998] GENERAL Reality Company. Homepage. Documento disponível na Internet no endereço http://www.genreality.com.

[I-GLASSES 1999] I-Glasses. I-O Display Systems Inc. http://www.i-glasses.com

[PINHO 1996] PINHO, Márcio S. Realidade Virtual como ferramenta de informática na educação. VII SBIE. Belo Horizonte - MG, Novembro, 1996.

[PINHO 1997] PINHO, Márcio S e KIRNER, Cláudio. Uma Introdução à Realidade Virtual. Mini-curso. X SIBGRAPI. Campos do Jordão, SP, Outubro, 1997.

[POLHEMUS 1999] Polhemus Company. http://www.polhemus.com

[PAUSH 1998] Paush, R. "Imersive Environments: research, applications and magic". Siggraph’98. Course notes., 1998.

[SHOHAM 1986] SHOHAM, Moshe. A text book of robotics 1: basic concepts. Tel Aviv, Israel : Kogan Page, 1986.

[WOO 1997] Woo, Mason et al. "OpenGL Programming Guide, The Official Guide to Learning OpenGL", Release 1, 1997.

[WOO 1998] Woo, M. "A visual introduction to OpenGl programming". Siggraph’98. Course notes., 1998.