Help understanding image detection

[thorrenat]

Hello Daniel,

First of all I want to thank you enourmously for this useful tool you designed, I am using it daily basis on my favourite game and after some learning I managed to automate it perfectly. Thanks.

In order to improve my scripts I've been trying to use image detection fuctions to enlarge my capabilities, sadly, I failed every time I tried.

I don't understand what I am doing wrong, I make objects, select mask, colour and at the time of running detections does nothing. It just walks through next action.

Would be possible for you to make a video or some extra documentation of how to use it?

Looking forward on hearing from you.
Best regards
R.

[DanielHarrod]

Hi R, That's great you have been able to figure it out. There's quite a bit of cool functionality in ATS. Including script design. I have a video in the works to cover many aspects of the functionality and script design. But it's a few weeks out. Here's a couple hints on object search that should get you running. Threshold is the relationship between the source and target image, each threshold will be different, because it's relative to the images being compared. Since it works at the individual RBG channel level, it's almost impossible to figure it out without using the Test button and tweaking it so that it meets the threshold at an acceptable level. The object search compares the two images, first it takes a screenshot, then it crops the image (this is the area to search). Then it takes the object and does math at every location on the image pixel by pixel and compares each possible location. It takes the highest score it finds after comparing every possible location, then checks to see if it meets the Threshold. If it meets the threshold then.... If you have a click event directly under the object search, it will click on the located image - centering the click event box size on that location then click there. Here's one On this event, the gold chest floats inside this range it goes up and down and scrolls left. So since it's only in that range, then I don't need to check anywhere else.  Here's my action, The size of the click area is how wide the clicks will be. Also since it's moving I have a -9 pixel offset.  It doesn't really matter where the box is, this is just configuring the click area and offset. This one and the above one work the same.  AHH...but..What if you don't want to click on the object. Instead you want to click somewhere else, here's the hack for that. (This is an option I need to build into the properties, but I haven't had any idea how to make it intuitive(ish)). Here's how you do it (if you don't want to click on the object you searched for). Object Search Event -> Event (empty or color/point) -> Click Action VS. Object Search Event -> Click Action **** The most important part. See the Object doesn't have the background included, that's because the background changes on every level and when the level scrolls. And I don't want the background in the calculation.  So this is:  Is not as good as this:  Object search takes considerably more CPU time to do the search than using color/points, so if you can do it with color/point that would be ideal or set the mask appropriately. (Also Mask isn't required) Here's another example, This one I want to click "Buy" if it's on the screen, and a few other icons, and scroll down the list, then scroll back to the top if I don't take any actions. First check that it's viable before doing object searches. That way it won't run every time.  Then If I see Buy, Click it. In the Village Group as it goes down the list, if it gets to the bottom, it performs scrolling actions if there's nothing found.  Let me know if this helps, Daniel

…

On Tue, Oct 25, 2022 at 5:28 AM thorrenat ***@***.***> wrote: Hello Daniel, First of all I want to thank you enourmously for this useful tool you designed, I am using it daily basis on my favourite game and after some learning I managed to automate it perfectly. Thanks. In order to improve my scripts I've been trying to use image detection fuctions to enlarge my capabilities, sadly, I failed every time I tried. I don't understand what I am doing wrong, I make objects, select mask, colour and at the time of running detections does nothing. It just walks through next action. Would be possible for you to make a video or some extra documentation of how to use it? [image: Studio] <https://user-images.githubusercontent.com/28702741/197749974-087c0e85-50ec-43c6-884c-ea59d762f22b.jpg> Looking forward on hearing from you. Best regards R. — Reply to this email directly, view it on GitHub <#46>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA7QNOJZ77WYLS3WUWUTC7DWE6Y5HANCNFSM6AAAAAARN2L3S4> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[thorrenat]

Hi Daniel,

Thanks fot this complete explanation, unfortunately example pictures were not loaded.
I will try all your proposals as soon as possible, and give you feedback.

Keep in touch

[DanielHarrod]

Hi R,

That's great you have been able to figure it out. There's quite a bit of
cool functionality in ATS. Including script design.

I have a video in the works to cover many aspects of the functionality and
script design. But it's a few weeks out.

Here's a couple hints on object search that should get you running.

Threshold is the relationship between the source and target image, each
threshold will be different, because it's relative to the images being
compared. Since it works at the individual RBG channel level, it's almost
impossible to figure it out without using the Test button and tweaking it
so that it meets the threshold at an acceptable level.

The object search compares the two images, first it takes a screenshot,
then it crops the image (this is the area to search). Then it takes the
object and does math at every location on the image pixel by pixel and
compares each possible location. It takes the highest score it finds after
comparing every possible location, then checks to see if it meets the
Threshold. If it meets the threshold then....

If you have a click event directly under the object search, it will click
on the located image - centering the click event box size on that location
then click there.

Here's one
On this event, the gold chest floats inside this range it goes up and down
and scrolls left. So since it's only in that range, then I don't need to
check anywhere else.

Here's my action, The size of the click area is how wide the clicks will
be. Also since it's moving I have a -9 pixel offset.

It doesn't really matter where the box is, this is just configuring the
click area and offset. This one and the above one work the same.

AHH...but..What if you don't want to click on the object.
Instead you want to click somewhere else, here's the hack for that. (This
is an option I need to build into the properties, but I haven't had any
idea how to make it intuitive(ish)).

Here's how you do it (if you don't want to click on the object you searched
for).
Object Search Event
-> Event (empty or color/point)
-> Click Action

VS.

Object Search Event
-> Click Action

**** The most important part.
See the Object doesn't have the background included, that's because the
background changes on every level and when the level scrolls. And I don't
want the background in the calculation.

So this is:

Is not as good as this:

Object search takes considerably more CPU time to do the search than using
color/points, so if you can do it with color/point that would be ideal or
set the mask appropriately. (Also Mask isn't required)

Here's another example, This one I want to click "Buy" if it's on the
screen, and a few other icons, and scroll down the list, then scroll back
to the top if I don't take any actions.
First check that it's viable before doing object searches. That way it
won't run every time.

Then If I see Buy, Click it.
In the Village Group as it goes down the list, if it gets to the bottom, it
performs scrolling actions if there's nothing found.

Let me know if this helps,

Daniel

[thorrenat]

Hello Daniel,

I tried some of the described steps, I am getting an error message "The type initializer for 'OpenCvSharp.Mat' threw an exception" every time I test or run .

Any idea how to solve that?

Thanks

[DanielHarrod]

Hi thorrenat,

This is a tricky one, as I wasn't able to reproduce it on any workstation that I have.

The application is having difficulty loading the DLL's in the install directory under /dll/x64 and /dll/x86

You might try running ATS as an administrator

...or making sure that the logged in user has security on those folders.

The other alternative is to run ATS from source code (eg install visual studio, run it from source code)

[thorrenat]

Hi Daniel,

That may explain something, I don't have such a folder:

The only folder is C:/Program Files/App Test Stidio/App Test Studio/bin

But no dll folder (not even in hiding folders)

Installed version 1.0.0.14 from 16 Jun 2022

[DanielHarrod]

Sorry, I didn't mean to say dll folder, I meant the bin folder.
I think it's not finding the dll's here.
You can copy them to the folder where AppTestStudio.exe is located and it will find them.

After the copy.

Let me know if this works.

[thorrenat]

This definetely made it work!!!
Thanks for your time and dedication helping me here.

I want to invite you to a 6 pack or something, how can I compensate you for all this effort?

[DanielHarrod]

That's awesome. Thanks for sticking with it. I'll try and see if this can be prevented in the future. Daniel

…

On Mon, Nov 14, 2022 at 10:15 AM thorrenat ***@***.***> wrote: This definetely made it work!!! Thanks for your time and dedication helping me here. I want to invite you to a 6 pack or something, how can I compensate you for all this effort? — Reply to this email directly, view it on GitHub <#46 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA7QNOJU4OV7H6JFEZTOSB3WIJQR5ANCNFSM6AAAAAARN2L3S4> . You are receiving this because you commented.Message ID: ***@***.***>

[thorrenat]

No, seriously, your patreon page is not working, how can I compensate you?

[DanielHarrod]

That's fixed too, I appreciate your support.