Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebase dev #700

Closed
wants to merge 55 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
95bece4
custom-fields: add config for hiding fields in upload/landing page
zzacharo Jun 2, 2023
fdaf967
communities: add /records and /members docs, replace id instances wit…
mitsosf Apr 21, 2023
738e9bb
authentication: document that info has to return external id
jrcastro2 May 11, 2023
3081a67
Oauth: add post log out docs
jrcastro2 Jun 19, 2023
7a9584b
files: conditional lock and versioning
kpsherva Jul 28, 2023
3865eaa
fixtures: Implement add-to-fixture command
Dec 11, 2023
9cd58a3
customize: update files locking function signature
slint May 21, 2024
9f954dd
all: fix grammar, typos, reformat tables
J4bbi Sep 29, 2023
80006c5
Added available sort options for all REST API queries that allow sorting
monotasker Mar 26, 2024
d5aa5a4
Minor fixes to default orders and link syntax
monotasker Apr 6, 2024
9e5f348
features: add access requests feature walk through
anikachurilova Apr 10, 2024
54d101f
pids: remove incomplete, unlinked, and outdated pids doc
tmorrell Jun 10, 2024
d3d5810
urns: update vor v12
tmorrell Jun 10, 2024
455caff
dois: v12 updates and config example
tmorrell Jun 10, 2024
3290bc2
notifications: add feature walk-through
rekt-hard Aug 1, 2023
6319d18
static pages: add html content sanitization config
Samk13 Jun 14, 2024
aede4f3
dois: fix typo
tmorrell Jun 17, 2024
1a5d613
Add parent doi explanation
tmorrell Jul 8, 2024
b63152a
custom fields: reorganize custom fields
tmorrell Jul 9, 2024
75cb3ee
docs: add notification section to customize and reference
rekt-hard May 9, 2023
ce9a1a5
docs: update notification reference and customization sections
rekt-hard Jun 1, 2023
2778a41
docs: refactor notification documentation into how-to, architecture a…
rekt-hard Aug 3, 2023
21a5830
upgrade: add steps to migrate to v12
utnapischtim Jul 16, 2024
e709b88
global: update workflows/tests.yml
utnapischtim Jul 16, 2024
8fd7079
fix: broken ci
utnapischtim Jul 16, 2024
784f54a
upgrade: add optional step
utnapischtim Jul 30, 2024
f880727
notifications+access request: primp for v12 release notes
fenekku Jul 12, 2024
b75e760
[WIP] release: v12.0
fenekku Jul 12, 2024
5be811e
[WIP] release: v12.0 with CHANGES.md consulted
fenekku Jul 15, 2024
677c17d
[WIP] release: v12.0 with cleanup
fenekku Jul 15, 2024
f8a15ba
[WIP] release: v12.0 - remove 12.0.1b1 release
fenekku Jul 16, 2024
4a0a3df
[WIP] release: v12.0 - add signposting
fenekku Jul 17, 2024
74d513c
[WIP] release: v12.0 - rebase cleanup
fenekku Jul 17, 2024
f72edcd
[WIP] release: v12.0 - add frontpage notice
fenekku Jul 17, 2024
492aa68
[WIP] release v12.0 - add line about configuration variables
fenekku Jul 18, 2024
768ef77
release-notes v12: add shared links limitation
fenekku Jul 24, 2024
97a5211
release-notes v12: update release date
fenekku Aug 1, 2024
a2befd7
release-notes v12: update CERN members in credits and fix typos
carlinmack Aug 2, 2024
43b18ef
release-notes v12: remove RoleNeed using id (not the case)
fenekku Aug 7, 2024
298635c
metadata: document parent.access.owned_by breaking change
fenekku Aug 13, 2024
b3a5063
release: add Trunk based Development Workflow
Samk13 Aug 14, 2024
8cabbc9
how-to: add example bash scripts for backing up stats indices
max-moser Jul 24, 2024
10ad294
how-to: reword warning about backing up all indices
max-moser Jul 25, 2024
cc54c11
release-notes v12: fix typos
fenekku Aug 21, 2024
acd5ba6
development: update assets watch instructions (#693)
Samk13 Aug 23, 2024
84a41f2
develop: document Python typing choices (#697)
slint Sep 16, 2024
d1d5bcd
v13: add draft release notes
ntarocco Aug 22, 2024
48ecf81
add missing files
ntarocco Aug 27, 2024
f1dd6d7
Reference: Add suggest API guide and custom index settings docs
sakshamarora1 Aug 6, 2024
cadaf9c
Reference: Update suggest API guide to reference edge_ngrams
sakshamarora1 Aug 16, 2024
a002de6
Reference: Add InvenioRDM version 13 in docs
sakshamarora1 Aug 20, 2024
8886462
Reference: small fixes to wording
carlinmack Aug 26, 2024
58fd930
names: add ORCiD public data sync
Jul 24, 2024
520cbd1
fix v10 links and add v13 links
ntarocco Aug 27, 2024
7fbce19
v13: add show packages version in admin panel
ntarocco Sep 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,12 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v4

- name: Set up Python 3.9
uses: actions/setup-python@v2
- name: Set up Python 3.12
uses: actions/setup-python@v5
with:
python-version: 3.9
python-version: 3.12

- name: Install dependencies
run: |
Expand All @@ -41,6 +41,6 @@ jobs:
run: mkdocs build --clean --strict -v

- name: Check links
uses: untitaker/hyperlink@0.1.27
uses: untitaker/hyperlink@0.1.32
with:
args: site/ --sources docs/
27 changes: 27 additions & 0 deletions docs/customize/authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -389,6 +389,29 @@ Properties marked with `True` (or omitted) in the precedence mask will be taken
from the authentication server user information payload if available, while properties marked
with `False` will be taken from the user input in the registration form.

### Defining post logout url

By default, upon logging out, the application will disconnect you only from itself. However, if you logged in through an external provider, logging out from the application will not automatically log you out from that provider. To address this, you can define a `logout_url` when setting up the remote apps:

```diff
_keycloak_helper = KeycloakSettingsHelper(
title="CERN",
description="CERN SSO authentication",
...
+ logout_url="your_app/logout",
),
)
```

After setting the `logout_url`, it is necessary to include the following configuration variable:

```python
SECURITY_POST_LOGOUT_VIEW = "/oauth/logout"
"""Required by invenio-oauthclient to be able to set logout urls for the remote apps."""
```

This will redirect to the appropriate `logout_url` for each of the enabled remotes in the instance.

## Security

For increased security, you should define the following in your `invenio.cfg`:
Expand Down Expand Up @@ -577,6 +600,10 @@ handlers = _keycloak_helper.get_handlers()
handlers["signup_handler"]["info_serializer"] = custom_info_serializer
```

!!! info "Mandatory external id"
When customizing an info endpoint, it is mandatory for a field to provide an external ID. This field must be named `external_id`.


### New OAuth plugins

If you need to implement your own OAuth plugin to enable integration with your OAuth provider,
Expand Down
60 changes: 56 additions & 4 deletions docs/customize/dois.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,28 @@ DATACITE_FORMAT = "{prefix}/inveniordm.{id}"
[Cool DOIs](https://doi.org/10.5438/55e5-t5c0) and why it might not be a
good idea.

#### Parent or Concept DOIs

_Introduced in InvenioRDM v12_

By default InvenioRDM will create two DOIs when an initial record is
published, and create one DOI each time a new version of the record is
published. The first DOI is the version DOI, which represents the specific
record that is published. The second DOI is the parent DOI, which represents
the concept of the record and will always resolve to the latest version.
This feature has been implemented in Zenodo for many years, and the concept DOI enables
researchers to cite something that won't change when they make changes to their
records.

![Concept DOI help-text](./img/concept_doi.png)

The parent DOI is optional and can be disabled by setting the following in
invenio.cfg:

```python
RDM_PARENT_PERSISTENT_IDENTIFIERS={}
```


#### OAI-PMH

Expand All @@ -78,9 +100,39 @@ The OAI-PMH server's metadata format ``oai_datacite`` that allows you to harvest
DATACITE_DATACENTER_SYMBOL = "CERN.INVENIORDM"
```

### Versioning and externally managed DOI

By default, InvenioRDM allows versioning for any DOI type - internally or externally managed. Internally managed DOI is a DOI which is given thanks to InvenioRDM feature which allows us to configure the DOI registration on DataCite (check #Enable DOI registration). The external DOIs are not minted by our instance and in some cases repository manager decides to disallow versioning of records identified by external DOI. To disable versioning for external DOIs you need to set:

```python
RDM_ALLOW_EXTERNAL_DOI_VERSIONING = False
```

#### Configuring DOI Behavior

You can change how DOIs work in InvenioRDM by adding modified configuration variables to `invenio.cfg`. For example, if you want to make DOIs *optional* you can add the following block to invenio.cfg

```python
RDM_PERSISTENT_IDENTIFIERS = {
# DOI automatically removed if DATACITE_ENABLED is False.
"doi": {
"providers": ["datacite", "external"],
"required": False,
"label": _("DOI"),
"validator": idutils.is_doi,
"normalizer": idutils.normalize_doi,
"is_enabled": providers.DataCitePIDProvider.is_enabled,
},
"oai": {
"providers": ["oai"],
"required": True,
"label": _("OAI"),
"is_enabled": providers.OAIPIDProvider.is_enabled,
},
}
```
You [can view the default configuration in invenio-rdm-records](https://github.com/inveniosoftware/invenio-rdm-records/blob/e64dd0b81757a391584e63d162d5e6caf6780637/invenio_rdm_records/config.py#L322)

## Known issues

- **Restricted records:** DOIs are registered for all records including restricted
records, thus metadata like titles, authors, description and more is sent to
DataCite Metadata registry where it is public. Provide an external DOI if
no DOI should be registered by InvenioRDM.
- **Restricted records:** Once a DOI is created, it cannot be fully removed from DataCite. Starting with v12, InvenioRDM will not register DOIs for restricted records. It will also hide a DOI from the DataCite Search is a record is changed from public to restricted. However that DOI will still resolve and metadata may be available to DataCite members.
35 changes: 35 additions & 0 deletions docs/customize/files-and-versioning.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Files and record versioning

The record versioning feature in InvenioRDM allows for the tracking and management of changes made to records' files over time. With this feature, every time a file of a record is updated, a new version of the record is created, capturing the changes made to the files.

This record versioning system enables researchers, data managers, and administrators to maintain a complete history of changes made to records, providing transparency and accountability in the data management process. Each version of the record includes a timestamp, indicating when the change was made, and the user who made the update.

By utilizing record versioning in InvenioRDM, institutions can maintain a reliable audit trail of all modifications made to records, enhancing data integrity, and contributing to good data management practices. This feature is an essential component of maintaining the accuracy, reproducibility, and long-term accessibility of research data and associated metadata in InvenioRDM.

In some cases repository manager might need to chose to disable the file versioning, for example.

- Storage Space: Record file versioning can result in an accumulation of multiple versions of files, which can consume significant storage space over time. In scenarios where storage resources are limited, disabling file versioning can help conserve storage and reduce costs.

- Simplicity: Some users may prefer a simpler and cleaner file management system without multiple versions of files. By disabling versioning, the system becomes more straightforward to manage, especially for users who do not require version history.

- Security and Privacy concerns: In certain cases, keeping multiple versions of files might pose security or privacy risks. Disabling versioning can help mitigate these risks by ensuring that only the latest version of the file is available, reducing the potential exposure of sensitive or outdated information.

- User Preferences: Some users may find versioning unnecessary for their specific use case and may prefer a file management system without version history. By allowing users to disable file versioning, InvenioRDM offers flexibility to cater to individual preferences.

- Performance: Maintaining version history can impact system performance, especially when handling a large number of files and records. Disabling versioning can help improve system responsiveness and speed, particularly in resource-constrained environments.

In order to provide custom rules for unlocking the file edition without creating a new version, you need to implement a callable which evaluates the conditions to return a bool value:

```python
def lock_edit_record_published_files(service, identity, record=None, draft=None):
"""Custom conditions for file bucket lock."""
if record:
is_external_doi = record.get("pids", {}).get("doi", {}).get("provider") == "external"
if is_external_doi:
return False
return True


RDM_LOCK_EDIT_PUBLISHED_FILES = lock_edit_record_published_files
"""Lock editing already published files (enforce record versioning)."""
```
Binary file added docs/customize/img/concept_doi.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/customize/img/custom_fields_example_deposit_form.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/customize/img/custom_fields_example_landing_page.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/customize/img/landing_page_fields_display.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/customize/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,4 @@ In the following sections, we cover customization opportunities that InvenioRDM
- [Subjects](vocabularies/subjects.md)
- [Users](vocabularies/users.md)
- [Internationalisation (i18n) and Localisation (l10n)](i18n-and-l10n.md)
- [Notifications](notifications.md) - customize content, recipients and backends of notifications
2 changes: 1 addition & 1 deletion docs/customize/look-and-feel/logo.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ institution's logo. We are going to change InvenioRDM's default logo to your log

## Step-by-step

**Step 1** - Take an *svg* file and copy it to your **local** static files.
**Step 1** - Take a *svg* file and copy it to your **local** static files.

We'll use the [invenio white logo](https://github.com/inveniosoftware/cookiecutter-invenio-rdm/blob/master/%7B%7Bcookiecutter.project_shortname%7D%7D/static/images/logo-invenio-white.svg) as an example:

Expand Down
Loading