Z a k e S t a h l

zake at (no spam) steelrabbit dot com




Enterprise Customer Technical Support Manager

"Maximize customer satisfaction through proficient identification and resolution of technical issues"

My ideal job is as an individual contributor, or manager of a team, in a software development or generative AI or AI as a software/platform company, doing external customer support or internal SecDevOps.

Hummingbird identification project.

This provides a demo of my engineering, AIPM, coding, ML, and, sensor to situational awareness pipeline development, abilities.

Background / Problem to solve:

I keep a clean, pure sugar-water, hummingbird feeder going year round. I enjoy watching the bird's territorial battles and sharing behaviors, seeing the seasonal changes.

Question became, can I use computer vision, machine learning tech, to track Humming birds?

To see what times of day, times of year, they visited, frequencies?

Maybe even to distinguish between individual birds, like (human) face recognition?


(Spin small carrots on left to view / hide solution detail.)
Motion capture software on Raspberry Pi with webcam; raw with no AI.

"Motion" software is a rich and mature FOSS package that is highly tune-able and on it's own got many pictures of Hummingbirds quite well. It also got thousands of jpgs of the bird feeder swinging in the wind, and people walking around in the background. Managing all the pictures generated becomes tedious.

Video pipeline from webcam to inference against model on Google Coral on Raspberry Pi; edge computing proof of concept with video.

    "GStreamer" is an amazingly versatile and powerful tool. I was able to stream video from webcam to inference engine running on Google Coral, and logged thousands of Humming birds at pretty high confidence rates. I was unable to satisfactorily 'tee' a success stream off to a jpg export however, so was not satisfied with this configuration.

    Here are some results, taken from a few weeks during Northern California spring 2023:

Time of Day Total Count of Birds Inferred
06:00 dawn 398
07:00 7
08:00 20
09:00 35
10:00 25
11:00 noon 106
12:00 87
13:00 29
14:00 39
15:00 55
16:00 101
17:00 87
18:00 dusk 153
19:00 7
20:00 1
21:00 1
22:00 thru 05:00 0

Inferenced Label Total Count of Birds Inferred
Calypte anna (Anna's Hummingbird) 285
Archilochus colubris (Ruby-throated Hummingbird) 234
Selasphorus rufus (Rufous Hummingbird) 41
Other Confidently Incorrect Birds 591

Code managing motion capture software and queuing images for labeling and notifications by inference against multiple models; edge computing proof of concept with jpgs.

Setup is: webcam on Pi; Python code running as daemon starts Motion hours before dawn, and stops it hours after dusk; code inferences captured images against animals model on Coral and labels them; as a lower priority non-matching images are tried against a general object model; total non-matches are purged some days later; hits for target birds trigger a "ntfy" notification. System has been successful, capturing hundreds of wonderful Hummingbird pictures!

humpi-webcam-feeder  58humdi

Pictures above show red and yellow Humming bird feeder, USB webcam, and (in upper right of first photo) upside down food storage container bungee-cabled to downspout. The "58" is percent humidity. In the enclosure is a Raspberry Pi 4 Model B and Google Coral USB.

Example photos just below show some recent visitors from the system. (Cat knows it can't catch the hummingbirds already.):

2023-11-16_14-49_228  2023-11-14_13-02_249.jpg  2023-11-12_12-55_94.jpg

Statistics for 2023-05-08 through 2023-11-17:
Passer_domesticus_House_Sparrow 201
Calypte_anna_Anna's_Hummingbird    114
Buteo_jamaicensis_Red-tailed_Hawk    78
Haemorhous_mexicanus_House_Finch    50
Archilochus_colubris_Ruby-throated_Hummingbird    17
Myiopsitta_monachus_Monk_Parakeet    16
Selasphorus_rufus_Rufous_Hummingbird    5
Archilochus_alexandri_Black-chinned_Hummingbird    2
Selasphorus_sasin_Allen's_Hummingbird    1

Humming bird face recognition - if you come to my feeder, you give up privacy...; next steps to implement.

Add a layer to neural net and then hand train on individual birds. If I can tell the difference by eye I can train the computer to do the same.

Send images right from webcam into AWS SageMaker and see what I can get out; next steps to implement.