new-project.qmd

---
date: "2023-05-26"
engine: julia
---

# New Julia Project {.unnumbered}
## First Time Julia Setup

::: {.callout-note}

You only have to follow these steps the first time after you download a new version of Julia and set it as your working version.

:::

Now that we have a better understanding of both Julia and the VSCode user interface, let's create a new Julia project.

First, open a fresh VSCode window.
If you already happen to be in a **workspace** (check in the "Explorer" panel), you can click on the "File" menu in the top left of the VSCode window, and then click "New Window".
Now, open the *Command Palette* (keyboard shortcut `Cmd + Shift + P` on Mac or `Ctrl + Shift + P` on Windows or Linux) and type "Julia: Start REPL".
Doing this should open up the integrated terminal with the Julia REPL running.
Type `pwd()` in the REPL and hit enter.
It should return your home directory (i.e. `/Users/<your-username>` on Mac or `C:\Users\<your-username>` on Windows).
From here we want to add a package called `DrWatson` to our Julia installation.
To do this, type `]` in the REPL and hit enter.
This enters the `Pkg` REPL, which is a special REPL for managing Julia packages.
Here you should see something like:

```{julia}
#| eval: false
(@v1.10) pkg>
```

Next, type `add DrWatson` and hit enter.
This will add the `DrWatson` package to your Julia installation, and will allow us to easily create new Julia projects in a consistent way that works well for epidemiology projects (among others).

::: {.callout-note}

If you're still wondering why we're bothering with the `DrWatson` package, you can read more about it [below](#drwatson-project-structure).

:::

## Creating a Folder for Your Projects

The first thing we need to do is create a folder that will house all our individual project folders.
To try and make your life easy, I would recommend you create this folder in your home directory, and call it ***Repos/*** (or something similar).
You can do this with Finder on Mac or File Explorer on Windows.

## Creating a Julia Project

Now let's create our first Julia project folder.
To do this, go back to the Julia `Pkg` REPL and hit backspace.
This will return you to the normal Julia REPL.
Now we need to navigate to the folder you just created.
To do this, type `cd("path/to/your/project/folder")` and hit enter.

From here, type `using DrWatson` and hit enter.
This will load the `DrWatson` package into your Julia session so you can use its functions.

To create the project folder, type `initialize_project("MyFirstProject")` and hit enter.
This will create a new project folder ***MyFirstProject/*** in the folder you navigated to earlier e.g., ***/Users/username/Documents/Repos/MyFirstProject/***.

Now you have a project folder, let's open it in VSCode.
In VSCode, click on the "File" menu in the top left of the VSCode window, and then click "Open Folder...".
From here, navigate to the project folder you just created and click "Open".
You VSCode window should refresh and you should now see the name of your project folder in the top center of the VSCode window (as well as in the top left of the "Explorer" panel).

Now, if you try to open the `Pkg` REPL again, you will see something different.
Instead it'll say:

```{julia}
#| eval: false
(MyFirstProject) pkg>
```

You're ready to start adding packages to your project!
I would recommend starting with following packages:

- `Revise`
- `DrWatson`
- `DataFrames`
- `DataFramesMeta`
- `CSV`
- `IJulia`
- `JuliaFormatter`
- `CairoMakie`

As before, to add a package, type `add <package-name>` and hit enter (you can add multiple packages at once by separating them with spaces e.g. `add <package-1> <package-2>`).

## Project Files

When you create a new Julia project, it creates two files in the project folder:

- ***Project.toml***: this is a TOML file that contains the names and versions of all the packages you have added to your project
- ***Manifest.toml***: this is a TOML file that contains the names and versions of all the packages you have added to your project, as well as the names and versions of all the packages that those packages depend on.
It is automatically generated by Julia, and you should not edit it manually.

Edit the ***Project.toml*** file to add your name in the authors field at the top of the file (you will need to create this), so it looks something like this:

```{toml}
name = "MyFirstProject"
authors = ["My Name"]

[deps]
CSV = "336ed68f-0bac-5ca0-87d4-7b16caf5d00b"
CairoMakie = "13f3f980-e62b-5c42-98c6-ff1f3baf88f0"
DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
DataFramesMeta = "1313f7d8-7da2-5740-9ea0-a2ca25f37964"
DrWatson = "634d3b9d-ee7a-5ddf-bec9-22491ea816e1"
IJulia = "7073ff75-c697-5162-941a-fcdaad2a7d2a"
JuliaFormatter = "98e50ef6-434e-11e9-1051-2b60c6c9e899"
Revise = "295af30f-e4ad-537b-8983-00126c2a3abe"
```

## Creating & Running a Julia Script

To create a Julia script in your project, click on the "File" menu in the top left of the VSCode window, and then click "New File".
Alternatively, you could use the keyboard shortcut `Cmd + N` on Mac or `Ctrl + N` on Windows or Linux.
This will open a new file in the VSCode editor, which you can then save.
Given we are using `DrWatson`, it is also good practice to save it in the ***scripts/*** folder.
You can save it with any file name you like, but it is good practice to save it with the `.jl` extension e.g., ***scripts/myscript.jl***.

At this point, you can start writing your Julia code.
Just make sure that the first two lines of the script are:

```julia
using DrWatson
quickactivate("MyFirstProject")
```

## Miscellaneous
### `DrWatson` Project Structure {#drwatson-project-structure}

It may not be immediately obvious why we need to use `DrWatson` to create our Julia projects.
In fact, we don't need to use `DrWatson` to create our Julia projects, but it does make our lives a lot easier as it provides a consistent project structure that is easy to navigate.

Firstly, it creates the following folders and files in your project folder (copied from the `DrWatson` documentation):

```
│projectdir          <- Project's main folder. It is initialized as a Git
│                       repository with a reasonable .gitignore file.
│
├── _research        <- WIP scripts, code, notes, comments,
│   |                   to-dos and anything in an alpha state.
│   └── tmp          <- Temporary data folder.
│
├── data             <- **Immutable and add-only!**
│
├── plots            <- Self-explanatory.
├── notebooks        <- Jupyter, Weave or any other mixed media notebooks.
│
├── papers           <- Scientific papers resulting from the project.
│
├── scripts          <- Various scripts, e.g. simulations, plotting, analysis,
│   │                   The scripts use the `src` folder for their base code.
│   └── intro.jl     <- Simple file that uses DrWatson and uses its greeting.
│
├── src              <- Source code for use in this project. Contains functions,
│                       structures and modules that are used throughout
│                       the project and in multiple scripts.
│
├── README.md        <- Optional top-level README for anyone using this project.
├── .gitignore       <- by default ignores _research, data, plots, videos,
│                       notebooks and latex-compilation related files.
│
├── Manifest.toml    <- Contains full list of exact package versions used currently.
└── Project.toml     <- Main project file, allows activation and installation.
                        Includes DrWatson by default.
```

In addition to the `{DrWatson}` produced directories, I like to create an `out/` directory where all analysis outputs are created, e.g., the ***sims*** shown in the structure above, rather than saving them to the `data/` directory, as `{DrWatson}` suggests.
The reason for doing this is that it provides a very clear separation of input and output files, where inputs should never be modified, and outputs are files that should always be able to deleted and recreated using the code in the scripts.
As a result, the final structure of the project looks like this:


```
│projectdir          <- Project's main folder. It is initialized as a Git
│                       repository with a reasonable .gitignore file.
│
├── _research        <- WIP scripts, code, notes, comments,
│   |                   to-dos and anything in an alpha state.
│   └── tmp          <- Temporary data folder.
│
├── data             <- **Immutable and add-only!**
│   └── exp_raw      <- Raw experimental data.
│
├── out              <- **Immutable and add-only!**
│   ├── exp_pro      <- Data from processing experiments.
│   └── sims         <- Data resulting directly from simulations.
│
├── plots            <- Self-explanatory.
├── notebooks        <- Jupyter, Weave or any other mixed media notebooks.
│
├── papers           <- Scientific papers resulting from the project.
│
├── scripts          <- Various scripts, e.g. simulations, plotting, analysis,
│   │                   The scripts use the `src` folder for their base code.
│   └── intro.jl     <- Simple file that uses DrWatson and uses its greeting.
│
├── src              <- Source code for use in this project. Contains functions,
│                       structures and modules that are used throughout
│                       the project and in multiple scripts.
│
├── README.md        <- Optional top-level README for anyone using this project.
├── .gitignore       <- by default ignores _research, data, plots, videos,
│                       notebooks and latex-compilation related files.
│
├── Manifest.toml    <- Contains full list of exact package versions used currently.
└── Project.toml     <- Main project file, allows activation and installation.
                        Includes DrWatson by default.
```

Above providing us with a useful project structure, `DrWatson` also provides us with a number of very helpful functions.

For one, it provides us with the `quickactivate()` function that allows us to activate our project and environment when we're working in a script, so we can easily import all the packages we need, regardless of where the script is saved.
This means that we don't need to worry about activating our project in the REPL before we start working in a script.
This may not sound meaningful, but it's very easy to forget to activate your project in the REPL before you start working in a script, and then you end up with a bunch of errors because you haven't actually imported the packages you need.
To use this, make sure your first two lines of your script are always:

```julia
using DrWatson
quickactivate("MyFirstProject")
```

<a name="drwatson-helpers">And secondly, it provides us with convenient directory paths to the folders in our project.</a>
For example, if we want to access the `data` folder in our project, we can use the `datadir()` function, which will return the path to the `data` folder.
Similarly, if we wanted to save a plot to the `plots` folder, we could use `save(plotsdir("plot_object.png"), plot_object)` function to save the plot in `plot_object` to the `plots` folder.

More details about the `DrWatson` project structure and associated functions can be found [here](https://juliadynamics.github.io/DrWatson.jl/stable/project/).

### Naming Files

When naming files and folders, it is important to be consistent and descriptive.
Ideally the name will contain no spaces, and you can use dashes to separate words.
See [this presentation](https://speakerdeck.com/jennybc/how-to-name-files) by Jenny Bryan for more information on naming files and folders, but in short:

- KISS (Keep It Simple Stupid): use simple and consistent file names
  - It needs to be machine readable
  - It needs to be human readable
  - It needs to order well in a directory
  - No special characters and no spaces!
- Use YYYY-MM-DD date format
- Use `-` to delimit words and `_` to delimit sections
  - i.e. ***2019-01-19_my-data.csv***
- Left-pad numbers
  - i.e. ***01_my-data.csv*** vs ***1_my-data.csv***
  - If you don’t, file orders get messed up when you get to double-digits