visualizers enable mouseClicked(); visualizers access their canvas

[katestange]

The visualizer should be able to use the mouseClicked() feature of p5.js.

Visualizers should be encouraged to use it as associated to a canvas, so the user has to click in the canvas (not just anywhere). In order for this to happen, the visualizer has to have access to its own canvas, as in the examples at the link above. See also #118.

[liammulh]

I believe this is part of @katestange's work on #115. (Or at least she made this issue while working on #115.)

[liammulh]

This blocks progress on #115.

[katestange]

This may be helpful: vue-p5 project by Kinrany

[gwhitney]

So the canvas you refer to is present in the p5 object as (in our case) this.sketch._renderer -- however, the privateness of this property as suggested in JavaScript by the name (beginning with underscore) is enforced in TypeScript, so there is no way to access the property without generating a TypeScript error. Presumably the reason the property is private is because the p5 folks feel it may change in the future. So I don't think it's a good idea to circumvent TypeScript here to access it.

So there are a couple of ways I have thought of to proceed:

1. The drawingContext (presuming you are now able to access that as I was per my recent comment in expose the canvas context to the visualizer #118) does contain a reference back to the canvas, which you can access via this.sketch.drawingContext.canvas. But note that obtaining it in this way yields a "raw" HTMLElement that you must manipulate using the basic JavaScript interfaces (as documented for example at https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement). It is lacking the fairly extensive convenience wrapper that p5 puts around such things. As a particular example, you cannot call .mouseClicked(handler) on this object, you have to call .addEventListener('click', handler). By no means terrible, but it doesn't exactly provide a simple/consistent programming environment for visualizer writers not coming from a web-techie background.

2. The canvas object you're thinking of, with the convenient .mouseClicked() method, is returned by the createCanvas() call, which of course has to happen in the setup function. The hiccup at the moment is that the CanvasArea Vue component (defined in Numberscope code) actually wraps the Visualizer's setup call with its own, and it's in that wrapper that the createCanvas() is being called (and its result is currently being discarded). So taking advantage of this point splits into two paths:

(2A) The wrapper could be changed to capture that canvas object and transmit it to the Visualizer. That's totally fine and doable. The only con to this approach is that it proceeds in the direction of making Numberscope more specifically tailored to p5-based Visualizers, when my preference would be to head in the other direction of making easier to add a Visualizer based on DataTables or three.js or whatever.

(2B) The CanvasArea wrapper could be simplified or perhaps removed altogether, so that the createCanvas would be happening in Visualizer code, where its return value could easily be saved for later use. The drawback here is that then there's common code that all Visualizers would need. But we could handle that by putting that boilerplate in VisualizerDefault that all visualizers derive from anyway. And this path moves in the direction of making the containing component less specific to p5, and hence easier to adapt in the future to other packages for visualization.

Let me know your thoughts on how you'd like to proceed here at the moment (we could refactor to another approach later).

[katestange]

This might be worth a live discussion, since I think I don't entirely understand the issues of specifically tailoring visualizer support to p5 vs. different platforms. Let's add it to the agenda for the meeting.

[liammulh]

At 2022-07-25 meeting, we agreed that 2B is the correct approach.

[katestange]

I'd like to move up the priority for this one slightly. I think our students may appreciate having mouseClicked() -- at least, I would. ;)

[gwhitney]

OK I haven't had the time to get to Zulip and now the Tuesday build has to be my top priority. So I will do Zulip as soon as I have all prep for Tuesday done, and then put this next on my agenda.

[katestange]

I would like to have keyPressed() also... I think at a first glance this is suffering the same issue.

[liammulh]

Assigning myself. I am going to try to implement the solution described in 2B in #120 (comment).

[katestange]

Hi @gwhitney this is blocking the NumberGlyph PR #231 which is otherwise supposedly ready; should we add this to the ToDo list for getting ready for Delft?

[gwhitney]

Yeah, I figured that was coming. Please do add it to the TODO discussion, and decide whether it's worth it to make a Project for the frontscope to prioritize things like I did with the backscope. Thanks. Unless @Vectornaut calls for help on the remaining backscope items (all of which are currently nominally in his court), I will make this my next Numberscope priority (but won't self-assign until I start in on it, in case someone else is just itching to give it a crack).

[gwhitney]

PS Whoever does work on it needs an example visualizer that would use the functionality, so if anyone is able to provide a specific example patch to an existing visualizer, or a new toy visualizer that would use this, please either provide a branch with that, or post a visualizer source file here, or something. It would be hard to implement without some example client code that would make use of it. Thanks!

[katestange]

The following branch should provide a simple tester that will check for mousePressed(), mouseReleased() and keyPressed(). It is also easily modifiable to add others. However, I don't have any way to test if it works! Note: I adapted the code from the p5.js reference pages. https://github.com/katestange/frontscope/tree/mouse_tester.

[katestange]

To use it ought to be self-evident; open numberscope and click on Mouse Tester visualizer, which has no parameters. The visualizer itself displays instructions.