Overall:
As part of the Colorado School of MINES FIRE program, I designed and prototyped a 4D haptic joystick working under Dr. Xiaoli Zhang in collaboration with PhD student Grant Zheng.
The mechanical design is based on other haptic joysticks on the market and previous research completed. Two geared DC motors with encoders control Θ and ɸ, and a servo powers a hydraulic syringe system to actuate in the z direction along the joystick. Vibration motors in the joystick top provide options to show spin, give positional guidance, or add to impact simulation. The DC motors can output roughly 4 N-m of torque each.
A Raspberry Pi 3 B+ is used to do computations and send out commands. Since there is a data processing aspect to this project, a Raspberry Pi allows for fast processing, signal outputs, and ease of use. The device is programmed in Python utilizing an object-oriented organization. There are two high-frequency motor drivers for the DC motors, a makeshift transistor diode PWM motor driver for the vibration motor, two current sensors for calibration, and a buck converter for power distribution.
We were designing towards simulating a tennis match with the joystick top being the ball. So a user would be holding onto the ball during the match as it went back and forth across the court. They would feel the impacts and kinematics the ball undergoes during the match.
We were able to achieve a successful simulation of the match with x-y data. There is a machine-learning part of this project that I am not a part of that is still working on getting the z-data of a given tennis match video.
Work is still being done to optimize the design, complete user studies, and write a paper for the 2025 IEEE Haptics Conference.
Mechanical Design:
All designed in SolidWorks and 3D printed on Bambu printers
Transmission Mechanisms
- Travel: 22° Θ and 36° ɸ
Motor shaft to transmission mechanism flange adapter
Bearing mounts
Motor mounts
Joystick base piece
Joystick syringe base
Joystick syringe top
Upper syringe z-axis actuation
- syringe plunger stabilized with metal rods
Joystick Top
- allows easy wire organization through the joystick
- integrates vibration motor in a minimal but effective way
Joystick syringe top piece
- houses syringe
- holes for wire routing and metal rod stabilizers
Electronics:
Raspberry Pi 3 B+
2x NOYITO H-bridge Motor Driver
2 x INA260 Current Sensor
1 x Transistor PWM Motor Driver
1 x Buck Converter
Control:
Programming:
Written in Python, organized using OOP.
Main scripts:
joystickcontrol.py: the base for running a simulation on the Haptic Joystick
joystickMotorControl.py: classes for each motor type with methods for each different desired function
data_for_demo.py: classes for data analysis and step creation
pid_controller.py: class for pid controller
rotary_encoder.py: class to decode rotary encoder readings (from the internet)
Available on GitHub once the paper is submitted
Simulation Technique:
To simulate the ball-racket impact, we tell the y-axis motor to go to an impossible positive angle and then an impossible negative angle, or vice versa depending on the path of the ball. This jolts the joystick in the direction of the ball's path then pulls it back rapidly before starting its next trajectory. This results in forces that the user can feel and relate to what they are seeing on the screen.
Haptics via Vibration Motors (alternate method):
Haptic Mapping (old design)
This is one possibility for the top of the joystick. This allows for vibration motors to vibrate when the ball is in a given quadrant of the court. There are many different options for using the vibration motors for mapping.
Since the top is quite small, the current mapping can be difficult to distinguish. This is in the beginning of its trial and error phase. The mechanical design is being reworked and optimized while the vibration and mapping pattern is being tested.
The vibration motors can also used to simulate impacts and to signal to the user how the ball is spinning.
Results:
The joystick was able to follow linear paths across the "court" with high precision and accuracy. There is a small offset of 60 ms between the goal position and the actual position. The red spikes are the impacts of the ball and racket. These create a jolt of the joystick that simulates the forces the ball feels to the user.
Here is an example of how the joystick simulates a tennis match. This video of the simulation is from the first generation of the joystick. At this time there was an issue with the data that was being processed into the joystick's path trajectories so it got off during the simulation.
Continued work includes optimization of haptics and code, completing user studies and writing the paper.
08/29/2024
