Request for Guidance on Using Clay Model to Identify Coffee Plantations

[cbalzani]

Hi, Clay team. Thank you so much for your efforts in bringing this tool to the world. It's fantastic work.

I need some help. Although I have a lot of experience with spatial data, I am not a developer. My knowledge of AI, Linux, Python, and almost all the basics needed to understand your model is very limited. That’s why I’m eagerly waiting for the Explorer.

If possible, I’d like some guidance on how to address my problem. I’ve gone through all the tutorials on Binder, but I didn’t quite understand how they connect to produce the result I’m looking for.

==> I’d like to use the Clay model to identify coffee plantation areas in a specific region.
How do I generate the embeddings to find coffee? Should I use the wall-to-wall tutorial for it?
After that, how do I export and use those embeddings to locate coffee-growing regions?

If you could point me toward the light at the end of the tunnel or the specific tutorials I should study to achieve my goal, that would be perfect.

I apologize for bringing up such a beginner-level topic in such a skilled community. I know these might be very silly questions.
Thank you so much for your attention :)

[rbavery]

"Embeddings" are the features of you can use as an input to fine-tuning a task specific classifier, detector, or segmentor. I recommend trying out the wall to wall example as a template for your problem, all the way to the last step where you can fine tune a classifier for coffee plantations: https://clay-foundation.github.io/model/tutorials/wall-to-wall.html#and-finally-some-fine-tuning

A classifier could label entire images as Coffee/ not coffee. In the wall to wall example an SVM is used but you could plug in a different model if you wanted to fine tune for object detection or segmentation. I advise starting simple with classification using the SVM in that wall to wall example.

[cbalzani]

Hi Ryan,

Thanks so much for your help! I’ll give it a try and definitely dive into the content to get more familiar with the basics.

Quick question though: once I’ve tuned the model with coffee embeddings using the wall-to-wall tutorial, what’s the next step to actually use it for finding coffee in other images? If you have any suggestions or tips on that as well, it would be fantastic!

Thanks again!

[rbavery]

you would run that finetuned model on your imagery, the model would produce text labels for each image that would be "coffee" or "no coffee". you would georeference those labels to the imagery vector footprint centroid and be able to have a map of points where coffee farms are approximately located.

[cbalzani]

Got it! So, just to confirm, that process would be separate from the wall-to-wall tutorial, right? Also, is it possible to export the finetuned model from the wall-to-wall and then run the model over images on Google Earth Engine? I’m more comfortable working with JavaScript. Thanks!

[rbavery]

Yeah the georeferencing would be a separate process and this repo doesn't have a tutorial for it that I'm aware of. Plenty of tutorials online though or LLMs can help here. If you have a need to work in python with geospatial data then this tutorial might be a good primer: https://carpentries-incubator.github.io/geospatial-python/

GEE doesn't support running pytorch models natively. you would need to use the GEE + Vertex AI integration I'm guessing but I'm not very familiar with it.

[rbavery]

You also might find TorchGeo to be a better starting point, they have tutorials that go over an intro to Pytorch, working with geospatial data for model training, and using TorchGeo to train models to solve different computer vision problems: https://torchgeo.readthedocs.io/en/latest/tutorials/getting_started.html