This application uses python 3.11.6 (currently 3.11.0 also works however if there are issues use .6 version)

use your package manager of choice to install the following libraries:

cv2
mediapipe
numpy
pandas
time
webbrowser
PIL.ImageGrab
keyboard
pyautogui
win32con
win32api

Potentially need to update pip if something doesn't install

When that is complete run the "HandDetecterClass.py" class

This application supports the following gestures:

Index and middle finger up: Open Google in the system's default browser
Only pinky up: Take a screenshot
Index finger and pinky up close selected program (Alt + F4)
Index, middle, and ring finger up: Close the current browser tab (Ctrl W)
Middle finger: [CENSORED]

All five fingers out: Activate mouse tracking

While in mouse mode:
Putting the index finger down in this mode will act as a click (Pinching your index with your thumb or curling your index finger down).
First (all fingers down): Exit mouse tracking mode.

The program can be stopped by entering "Ctrl + C" on the console or "Esc" on the camera vision window.

The mouse tracking tracks "Index finger mcp" as depicted in "THE HAND.png"

a finger is considered down if the corresponding "dip" is closer to the wrist than the "pip" as described by THE HAND


Failures:
Not many to speak of, we experimented with a couple of mouse options but found one we liked in win32

Successes:
Using numpy and some quick math to easily and quickly determine whether a finger is "up" or not

Improvements:
We would like to refine the precision and ease of use with mouse mode and incorporate swiping gestures.

Future work:
Integrate linear classifier and cleaner gestures, also process rejections better such as background hand motions and mistaken gestures