Add Dev Container configuration for easier and more reproducible setup.

[wolfmanstout]

See README.markdown changes for documentation.

[wolfmanstout]

As noted in #699, we could try a later Python version (presumably).

[davidebbo]

I have not used dev containers, but it seems like an interesting approach. When I get some cycles, I'll try to play with it.

I assume that it is orthogonal to our need to move to a venv model, per #676 (we should do that as well).

[wolfmanstout]

Yes, this is orthogonal to venv (we could easily set up the venv in dev.Dockerfile, and in fact this would be another example of Dev Containers saving setup time).

I will note that a venv isn't really necessary if you are using a Dev Container, because you are only using the Python instance for a single project so there is no reason not to install packages globally. But if we want to keep everything consistent, we can easily do so.

[wolfmanstout]

FYI this may be broken if you sync this to head instead of trying my branch directly. It's related to tours database changes since the main docker image that this references. I need to investigate further and make sure my approach is as robust as possible to DB changes.

[davidebbo]

I will note that a venv isn't really necessary if you are using a Dev Container, because you are only using the Python instance for a single project so there is no reason not to install packages globally. But if we want to keep everything consistent, we can easily do so.

Yes, good point. But since not everyone will use Dev Containers, it's still best to proceed with the shift to venv.

[wolfmanstout]

FYI this may be broken if you sync this to head instead of trying my branch directly. It's related to tours database changes since the main docker image that this references. I need to investigate further and make sure my approach is as robust as possible to DB changes.

I would consider this issue resolved. I did a fresh rebuild of my container and the problem went away. I think what happened was that my table files and the database got in an inconsistent state; maybe I accidentally killed the server at a bad time at one point.

Consider this ready for review/merging.

[davidebbo]

Still trying to grok the model.

and another (web) with the MySQL database

Ok, so only the web container is running mysql. So when running web2py.py directly in the dev container, it's actually connecting to the mysql instance in the web container, thereby going cross container? In other words, they are sharing the DB.

[wolfmanstout]

Yes, that is all correct. I originally tried a one container configuration, extending from the prod Dockerfile. That didn't work; it was missing dev tools and the vscode integration was very buggy. I still wanted to reuse as much of that Dockerfile as possible so I split out the dev container. This is apparently a standard practice when working with databases, e.g. see this template: https://github.com/devcontainers/templates/tree/main/src/postgres

I chose a Node.js base image and added Python as a feature. I could've done the reverse too. Another alternative is we could use the universal Linux image which has both (https://github.com/devcontainers/images/tree/main/src/universal), but we lose control over the Python and Node.js versions (FWIW the current versions match ours). A couple advantages to that are that the builds would be faster (adding a feature like Python is slow) and storage would be free on GitHub Codespaces (this is their default image and they don't charge for storage of containers based on this, except for additional files added). I'm open to that change if folks prefer.

[davidebbo]

Great, thanks for confirming. I think we should just start with that and see how it goes.

About the DB, I exposed the port in the docker-compose to be able to open it from MySQL workbench (running on Windows). e.g.

    ports:
      - "8080:80"
      - "3307:3306"

I went with 3307 locally to avoid conflicting with my Ubuntu MySQL. Might be worth mentioning that in the doc, and maybe just add that to the default docker-compose.

[wolfmanstout]

Done and documented.

[davidebbo]

@hyanwong my take is that this is ok to merge. It's a new option that does not affect the standard dev model, so even though we might still need some small tweaks, it will help move things forward.

[lentinj]

I think it'd be a useful option to have, and definitely an easier setup process than the current "install a bunch of stuff, get the docker image and extract the database" process we have.

There's some tidying I think should be done first though, and I'm still not sold on why we need 2 OZtree directories, but I'm running at half-caffeination so apologies if I've just not grasped what's going on.

[wolfmanstout]

I think it'd be a useful option to have, and definitely an easier setup process than the current "install a bunch of stuff, get the docker image and extract the database" process we have.

There's some tidying I think should be done first though, and I'm still not sold on why we need 2 OZtree directories, but I'm running at half-caffeination so apologies if I've just not grasped what's going on.

I'm happy to elaborate on anything. When you say "2 OZtree directories", are you referring to the use of 2 containers (dev and web) or the fact that the original OZTree directory is mounted in the dev container? The only reason I do the latter is to ease file copying of the database files. For the former, this is necessary today because the upstream container is needed to get the database information, and it is not suitable to be extended as a dev container by itself (I originally tried this and it had all kinds of problems; much better to start from a base container image designed for dev container usage). Also, having a separate container for the database is actually a common practice and there are official templates that do this (I linked to one earlier in my conversation with David).

[hyanwong]

Merged. Thanks @wolfmanstout !