customization: instructions for changing menus

docs/customize/look-and-feel/menus.md

@@ -0,0 +1,150 @@
+# Change Menus/Tabs
+
+For navigation, InvenioRDM often uses menus or tabs. For example, one such menu is the user-dropdown menu:
+
+![User-settings menu with its 5 default entries](./img/settings-menu.png)
+
+And one such set of tabs is at the top of the user-dashboard:
+
+![User's personal dashboard with three tabs labeled (left-to-right) "Uploads", "Communities", and "Requests"](./img/user-dashboard.png)
+
+In the codebase, menus and sets of tabs like those are referred as *menus*, so we do so here for the remainder of this how-to as well.
+Their options (e.g., "Uploads", "Profile") are referred as *submenus*.
+
+These *menus* can be configured in the following ways:
+
+- which submenus to show
+- what title-text to show in those submenus
+- which endpoint to request when clicking them
+- how to dynamically show/hide them (e.g. depending on currently logged-in user)
+
+## Configurable Properties
+
+Properties of existing submenus can be overridden,
+properties of newly added submenus can be freely chosen.
+All submenus share the following properties:
+
+| Name           | Example values                       | Notes                                               |
+|----------------|--------------------------------------|-----------------------------------------------------|
+| `endpoint`     | `"invenio_app_rdm_users.uploads"`    | flask-endpoint requested when clicking this submenu |
+| `order`        | `1`, `2`, `10`, `-1`, `1.5`          | lower ordered submenus are shown further left       |
+| `text`         | `_("Uploads")`, `_("Communities")`   | title-text shown on submenus                        |
+| `visible_when` | `flask_menu.menu.CONDITION_TRUE`     | when to show this submenu<br />- must be of type `callable[[], bool]`<br />- `CONDITION_TRUE` is just a named `lambda: True`<br />- use `CONDITION_FALSE` to hide a submenu |
+| ...            | ...                                  | in addtion to the above properties, all arguments for `flask_menu.menu:MenuNode.register` are configurable |
+
+For example, consider the user-dashboard pictured above.
+Its *Uploads* submenu has the following default configuration:
+```python
+{
+    "endpoint": "invenio_app_rdm_users.uploads",
+    "order": 1,  # small number as to show this submenu left-most
+    "text": _("Uploads"),
+    "visible_when": flask_menu.menu.CONDITION_TRUE,  # always show this submenu
+    ...,  # advanced configuration properties omitted for brevity
+}
+```
+
+## Add new submenus to existing menu
+
+Existing menus can be extended by custom-configured submenus.
+To add a new submenu to an existing menu:
+
+1. Get the name of the to-be-added-to menu from the [list of menus](#list-of-menus)
+  For example, the user-dashboard is named `"dashboard"` (most names of menus/submenus are straightforward).
+2. Write a function that registers the new submenu
+  ```python
+  # ext.py  # ext.py is commonly used, you may use another file though
+
+  from flask_menu import current_menu
+  from invenio_i18n import lazy_gettext as _  # for translations
+
+  def finalize_app():
+      # get the user-dashboard menu via its name "dashboard":
+      user_dashboard_menu = current_menu.submenu("dashboard")
+
+      # register a new submenu to the user-dashboard:
+      user_dashboard_menu.register(
+          "name-of-submenu",
+          endpoint="my_blueprint.my_endpoint",
+          order=4,  # the three already existing submenus have `order` 1 through 3
+          text=_("Title of new Submenu"),  # could also use an untranslated bare string
+          # note: could add other properties here, otherwise their default is used
+          # note: `visible_when` defaults to always visible
+      )
+
+      ...  # could .register more submenus here (or do so in another package)
+  ```
+3. Register your function with the `invenio_base.finalize_app` entrypoint.
+  This will make your function be called at the app-finalization build-step.
+  For example, when using setuptools' `setup.cfg` with your python package, add:
+  ```ini
+  # setup.cfg
+
+  [options.entry_points]
+  invenio_base.finalize_app =
+      my_package_name = my_package_name.ext:finalize_app
+  ```
+
+!!! info "For entrypoints to take effect"
+    For entrypoint changes to be picked up, you will need to reinstall the python package.
+    *This is necessary even if the package is installed editably!*
+
+    After entrypoint changes are picked up,
+    you will further need to restart the server for changes to take effect:
+    ```shell
+    <CTRL+C>
+    invenio-cli run
+    ```
+
+## Modify existing submenus
+
+The defaults of submenus' properties are selectively overridable via config-variables.
+To modify an existing submenu's properties:
+
+1. Find the name of the corresponding override-variable in the [list of menus](#list-of-menus)
+  For example, the override-variable for the user-dashboard is named `USER_DASHBOARD_MENU_OVERRIDES`.
+2. Find the name of the to-be-overridden submenu in that same [list of menus](#list-of-menus)
+  For example, the user-dashboard has a submenu for communities.
+  This submenu is named (obviously enough) `"communities"`.
+3. Add to your `invenio.cfg`:
+  ```python
+  # invenio.cfg
+
+  from flask_menu.menu import CONDITION_FALSE
+
+  USER_DASHBOARD_MENU_OVERRIDES = {
+      "communities": {
+          "visible_when": CONDITION_FALSE,
+          # other properties will be left unchanged
+      }
+      # other submenus will be left unchanged
+  }
+  ```
+
+The above example hides the *Communities* submenu from the user-dashboard menu
+by overriding its `visible_when` property.
+For other overridable properties see [configurable properties](#configurable-properties).
+
+!!! info "Have you tried to turn it off and on again?"
+    You will need to restart the server for changes to take effect:
+    ```shell
+    <CTRL+C>
+    invenio-cli run
+    ```
+
+!!! warning "On overriding user-added submenus: DON'T"
+    Currently, (sub)menu-overrides are adopted at app-finalization.
+    Users adding their own submenus (as described above) is also done at app-finalization.
+    Hence attempting to override submenus added by (other) users depends on
+    loading order of app-finalization entrypoints and might break anytime.
+
+## List of menus
+
+The following list of menus is non-exhaustive.
+Not all menus are overridable, but all menus can be extended by additional submenus.
+
+| Image | Description | Name of menu and its submenus | Name of override-variable |
+|-------|-------------|-------------------------------|---------------------------|
+| ![User's personal dashboard with three tabs labeled (left-to-right) "Uploads", "Communities", and "Requests"](./img/user-dashboard.png) | Dashboard on user's personal page | `"dashboard"`<br />├─`"uploads"`<br />├─`"communities"`<br /> └─`"requests"` | `USER_DASHBOARD_MENU_OVERRIDES` |
+| ![Communities dashboard with its 6 default tabs](./img/communities-dashboard.png) | Dashboard on communities page | `"communities"`<br />├─`"home"`<br />├─`"search"`<br />├─`"requests"`<br />├─`"submit"`<br />├─`"members"`<br />├─`"settings"`<br />├─`"curation_policy"`<br />└─`"about"` | No associated override-variable |
+| ![User-settings menu with its 5 default entries](./img/settings-menu.png) | User settings menu | `"settings"`<br />├─`"profile"`<br />├─`"change_password"`<br />├─`"security"`<br />├─`"notifications"`<br />├─`"oauthclient"`<br />└─`"applications"` | No associated override-variable |

docs/customize/metadata/custom_fields/records.md

@@ -283,7 +283,7 @@ You should add the `my_template.html` file in the `my-site/templates` folder in
 - `field_value`: the value of the field, as it is stored in the record after the UI serialization i.e. what is returned from the `ui_field` method when you [define your custom field](../../../develop/howtos/custom_fields.md).
 - `field_cfg`: the UI configuration for that specific field as it is defined in the `RDM_CUSTOM_FIELDS_UI` config.
 
-See the example in the [How-to](../../../develop/howtos/custom_fields.md#define-the-template-for-the-record-landing-page).
+See the example in the [How-to](../../../develop/howtos/custom_fields.md).
 
 ### Search
 

docs/customize/vocabularies/names.md

@@ -17,7 +17,7 @@ A _Name_ record contains:
   are present they will overwrite `name`.
 - A list of `identifiers`, composed by their identifier value and scheme.
   The scheme can potentially be autocompleted if it is known by the _idutils_
-  library (e.g. ORCiD).
+  library (e.g. ORCID).
 - A list of `affiliations`, which can be represented by its `name` or, if it
   belongs to the _Affiliations_ vocabulary, by its `id`.
 
@@ -96,7 +96,7 @@ invenio vocabularies update \
 ### Creating a `names.yaml` file
 
 The Names vocabulary has been implemented with the
-[ORCiD public dataset](https://orcid.figshare.com/articles/dataset/ORCID_Public_Data_File_2021/16750535?file=31020067)
+[ORCID public dataset](https://support.orcid.org/hc/en-us/articles/360006897394-How-do-I-get-the-public-data-file)
 as a possible source to import entries from. This means that the functionality
 to **read** entries from this format is already available. For example, you
 can use the `vocabularies convert` command to convert this dataset into a YAML
@@ -112,7 +112,7 @@ invenio vocabularies convert \
 Alternatively, you can simply import it directly:
 
 !!! warning "Long and blocking operation"
-    Note that the import process is done synchronously and the ORCiD dataset is
+    Note that the import process is done synchronously and the ORCID dataset is
     very large. Therefore, this operation can take a long time.
 
 ```bash

docs/develop/howtos/custom_code.md

@@ -24,7 +24,7 @@ To generate the site folder, you will need to select option `1 - yes` (this is a
 ```python hl_lines="3"
 [packages]
 ...
-my-site = {editable="True", path="./site"}
+my-site = {editable=true, path="./site"}
 ```
 
 This means that the site folder will be installed as a package with the name `my-site`, and it is editable. This package now works as any other package installed in your instance (`invenio-app-rdm`, `invenio-communities`, etc.), allowing you to customize your instance and create new views and features without adding a separate package manually.

docs/develop/howtos/restrict_access.md

@@ -0,0 +1,52 @@
+# How to restrict access to pages
+
+Sometimes it can be desirable to live by the motto "better safe than sorry", especially regarding potentially sensitive features like the administration panel (enabled in InvenioRDM v12).
+
+This guide briefly describes how to narrow down access to subsets of the system.
+
+
+## Restricting access for IP ranges via `nginx`
+
+While most features in InvenioRDM are guarded by configurable permission policies, this isn't necessarily always the case.
+For these exceptions, as well as extra precautions generally, it can be beneficial to restrict access on an `nginx` level.
+
+!!! info "Current exceptions"
+    At the time of writing, one of these exceptions is the administration panel which has a hard-coded check for the `administration-access` action.
+
+An access restriction based on the client's IP address can be put into place via the `nginx` configuration, e.g. by adding nested `location` directives in the existing configuration:
+
+```nginx
+location / {
+    uwsgi_pass ui_server;
+    include uwsgi_params;
+    # ... your configuration for the UI paths ...
+
+    # restrict access to the administration panel UI to your network only
+    location /administration/ {
+      # action directives like `uwsgi_pass` aren't inherited like other configs
+      uwsgi_pass ui_server;
+
+      # allow your networks (replace with your IP ranges)
+      allow 128.130.0.0/15;
+      allow 192.35.240.0/22;
+      allow 2001:629::/32;
+      # etc.
+
+      # also allow localhost and private networks (e.g. for local access through Docker)
+      allow 127.0.0.1/8;
+      allow ::1/128;
+      allow 10.0.0.0/8;
+      allow 172.16.0.0/12;
+      allow 192.168.0.0/16;
+      allow fd00::/8;
+
+      # disallow anybody else
+      deny all;
+    }
+}
+```
+
+!!! info "The `uwsgi_pass` directive doesn't get inherited"
+    Note that the `uwsgi_pass` directive is part of a [class of directives that do not get inherited in nested locations](https://forum.nginx.org/read.php?2,243488,243488) and thus has to be specified explicitly again.
+
+Restricting access to API endpoints follows a similar schema, but in the `location /api` block and with `uwsgi_pass api_server` instead.

docs/develop/index.md

@@ -53,6 +53,7 @@ Step-by-step guides on how to perform certain tasks:
 - [Fix a vulnerability](howtos/security-fix.md)
 - [Test emails locally](howtos/dev_email.md)
 - [Migrate legacy routes](howtos/route_migration.md)
+- [Restrict access to pages](howtos/restrict_access.md)
 
 ## Architecture
 

docs/reference/cli.md

@@ -480,3 +480,7 @@ invenio rdm rebuild-all-indices -o users,communities,records,requests,request_ev
 if you don't specify services, The following services will be reindexed:
 
 `users, groups, domains, communities, members, records, record-media-files, affiliations, awards, funders, names, subjects, vocabularies, requests, request_events, oaipmh-server`
+
+Note that the users, groups, and members use bulk indexing and rely on celery running. They will not be reindexed if celery is not running.
+
+This command does not impact usage statistics indexes. You need to manually restore statistics indexes [from a backup](../develop/howtos/backup_search_indices.md).

docs/reference/metadata.md

@@ -117,7 +117,7 @@ External PIDs are persistent identifiers managed via [Invenio-PIDStore](https://
 with external registration services.
 
 Persistent identifiers are globally unique in the system, thus you cannot have two records
-with the same system-managed persistent identifier (see also [Metadata > Identifiers](#identifiers-0-n)).
+with the same system-managed persistent identifier (see also [Metadata > Identifiers](#alternate-identifiers-0-n)).
 
 You can add a DOI that is not managed by InvenioRDM by using the provider `external`. You are not able to add `external` DOIs that have a prefix that is configured as part of a different PID provider.
 
@@ -1091,7 +1091,7 @@ Example:
     IIIF links are only returned for files who are compatible with IIIF.
     Those formats are defined by the `IIIF_FORMATS` configuration variable.
     By default _gif_, _jp2_, _jpeg_, _jpg_, _png_, _tif_, and _tiff_.
-    
+
 ### Default preview (0-1)
 
 The default preview field names the filename of the file which should by default

docs/reference/rest_api_communities.md

@@ -779,7 +779,7 @@ Content-Type: application/json
 |----------------|--------|----------|-----------------------------------------------------------------------------------------------|
 | `accept`       | string | header   | - `application/json` (default)<br />- `application/vnd.inveniordm.v1+json`                    |
 | `community_id` | string | path     | ID of the community.                                                                          |
-| `start_date`   | string | body     | Required, datetime in iso format. Community will be featured from this point in time onwards. |
+| `start_date`   | string | body     | Required, ISO 8601 DateTime format in UTC (YYYY-MM-DDTHH:MM:SS.ssssssZ). Community will be featured from this point in time onwards. |
 
 
 **Request**