Simplify data entry forms

[marlonbaeten]

Fixes #850

This PR is an overhaul of the state management for the data entry forms.

It introduces a "reducer" to centralize state manipulation and simplify the data flow using events.

Apart from the central form state, the local form hooks and components are cleaned up, in particular the number of hard-to-reason-about useEffect and useRef hooks are removed.

See frontend/app/component/form/data_entry/state for the hooks, context and providers related to the data entry state.

Notes:

• This refactor is the MVP for less complex state management in the data entry forms, its not perfect, especially in the util functions there is still some refactoring to be done.
• This PR introduces the reducer and simplifies the central state; however, the central state is not perfectly compact as it should be.
• We focussend on rewriting the most complex parts and getting it back into main ASAP
• See https://react.dev/reference/react/useReducer for documentation about the useReducer

Other notes on React state management:

The data flow we try to adhere to

┌────> State ──> View ───┐
│                        │
│                        │
└──────── Events <───────┘

In our case events are triggered from the UI, which are dispatched to the central reducer. The reducer updates the otherwise immutable state object. The React components render this immutable state as a UI.

Don't overuse useRef

From the React documentation:

When to Use Refs

There are a few good use cases for refs:

• Managing focus, text selection, or media playback.
• Triggering imperative animations.
• Integrating with third-party DOM libraries.

Avoid using refs for anything that can be done declaratively.

Don’t Overuse Refs

Your first inclination may be to use refs to “make things happen” in your app. If this is the case, take a moment and think more critically about where state should be owned in the component hierarchy. Often, it becomes clear that the proper place to “own” that state is at a higher level in the hierarchy. See the Lifting State Up guide for examples of this.

Don't overuse useEffect

From the React documentation:

You Might Not Need an Effect

Effects are an escape hatch from the React paradigm. They let you “step outside” of React and synchronize your components with some external system like a non-React widget, network, or the browser DOM. If there is no external system involved (for example, if you want to update a component’s state when some props or state change), you shouldn’t need an Effect. Removing unnecessary Effects will make your code easier to follow, faster to run, and less error-prone.

Strive for simple components with unidirectional data flow

• Event handlers should trigger state updates
• The events objects passed from the DOM are excellent references to DOM elements and their values -> we don't need useRef
• Only use useEffect to preload API data, or trigger explicit side-effects
• Keep state minimal and calculate derived state in the render component
• The useMemo and useCallback hooks should only be used if they prevents expensive calculations - in Abacus this is almost never the case!

[codecov]

Codecov Report

Attention: Patch coverage is 95.42816% with 63 lines in your changes missing coverage. Please review.

Project coverage is 92.78%. Comparing base (b819a8c) to head (ed2d539).

✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
.../component/form/data_entry/DataEntryNavigation.tsx	85.27%	19 Missing ⚠️
.../form/data_entry/state/useInitialDataEntryState.ts	79.01%	17 Missing ⚠️
...end/app/component/form/data_entry/state/actions.ts	90.47%	10 Missing ⚠️
...m/data_entry/candidates_votes/useCandidateVotes.ts	92.94%	6 Missing ⚠️
...end/app/component/form/data_entry/state/reducer.ts	97.76%	3 Missing ⚠️
...ata_entry/candidates_votes/CandidatesVotesForm.tsx	96.66%	2 Missing ⚠️
...onent/form/data_entry/state/useDataEntryContext.ts	87.50%	2 Missing ⚠️
frontend/lib/util/format.ts	87.50%	2 Missing ⚠️
.../component/form/data_entry/state/dataEntryUtils.ts	96.00%	1 Missing ⚠️
...nt/form/data_entry/state/useDataEntryNavigation.ts	97.14%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #964      +/-   ##
==========================================
+ Coverage   92.06%   92.78%   +0.72%     
==========================================
  Files         281      288       +7     
  Lines       17123    17388     +265     
  Branches     1503     1538      +35     
==========================================
+ Hits        15764    16134     +370     
+ Misses       1256     1151     -105     
  Partials      103      103              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

Sigrid maintainability feedback

⚠️ Your code did not improve maintainability towards your objective of 3.5 stars

Show details

Sigrid compared your code against the baseline of 2025-03-05.

👍 What went well?

You fixed or improved 1 refactoring candidates.

Risk	System property	Location
🟡	Unit Size (Improved)	frontend/app/component/form/data_entry/check_and_save/CheckAndSaveForm.tsx CheckAndSaveForm.tsx.CheckAndSaveForm()

👎 What could be better?

Unfortunately, 76 refactoring candidates were introduced or got worse.

Risk	System property	Location
🔴	Duplication (Introduced)	frontend/app/component/form/data_entry/candidates_votes/useCandidateVotes.ts line 90-106 frontend/app/component/form/data_entry/voters_and_votes/useVotersAndVotes.ts line 67-83
🔴	Duplication (Introduced)	frontend/app/component/form/data_entry/candidates_votes/useCandidateVotes.ts line 85-101 frontend/app/component/form/data_entry/differences/useDifferences.ts line 56-72
🔴	Duplication (Introduced)	frontend/app/component/form/data_entry/candidates_votes/CandidatesVotesForm.tsx line 29-38 frontend/app/component/form/data_entry/differences/DifferencesForm.tsx line 23-32
🔴	Duplication (Introduced)	frontend/app/component/form/data_entry/voters_and_votes/useVotersAndVotes.ts line 45-60 frontend/app/component/form/data_entry/differences/useDifferences.ts line 34-49 frontend/app/component/form/data_entry/candidates_votes/useCandidateVotes.ts line 57-72
🔴	Duplication (Introduced)	frontend/app/component/form/data_entry/differences/DifferencesForm.tsx line 27-33 frontend/app/component/form/data_entry/voters_and_votes/VotersAndVotesForm.tsx line 28-34
🔴	Duplication (Introduced)	frontend/app/component/form/data_entry/state/dataEntryUtils.ts line 293-298 frontend/app/component/form/data_entry/state/dataEntryUtils.ts line 315-320 frontend/app/component/form/data_entry/state/dataEntryUtils.ts line 330-335 + 2 occurrences
🔴	Duplication (Introduced)	frontend/app/component/form/data_entry/candidates_votes/CandidatesVotesForm.tsx line 33-38 frontend/app/component/form/data_entry/voters_and_votes/VotersAndVotesForm.tsx line 28-33
🔴	Unit Size (Introduced)	frontend/app/component/form/data_entry/state/dataEntryUtils.ts dataEntryUtils.ts.getInitialFormState(Required<Election>,Partial<FormState>)
⚫️		+ 68 more

📚 Remaining technical debt

5 refactoring candidates didn't get better or worse, but are still present in the code you touched.

View this system in Sigrid to explore your technical debt

⭐️ Sigrid ratings

System property	System on 2025-03-05	Before changes	New/changed code
Volume	5.1	N/A	N/A
Duplication	4.3	2.2	2.5
Unit Size	2.4	3.5	1.8
Unit Complexity	3.4	1.3	2.2
Unit Interfacing	3.2	5.3	3.2
Module Coupling	3.6	5.5	3.0
Component Independence	2.6	N/A	N/A
Component Entanglement	3.0	N/A	N/A
Maintainability	3.5	3.4	2.5

💬 Did you find this feedback helpful?

We would like to know your thoughts to make Sigrid better.
Your username will remain confidential throughout the process.

• ✅ Yes, these findings are useful
• 🔸 The findings are false positives
• 🔹 These findings are not important to me

---

View this system in Sigrid

[lkleuver]

If I get to the Voters and Votes page, then return to the Recounted page, change the radio button, and click in the nav to go the Voters and Votes page, I don't get a pop-up to save or discard changes.

Is this something that can be added to model based testing? changes popup inbetween sections?
It should be fixed now, recounted was missing the flag hasChanges

[lkleuver]

On the Differences page, if the error "Je kan alleen verder als je het het papieren proces-verbaal hebt gecontroleerd." is shown and the accept warnings-checkbox is ticked, changing the data in the form does not untick the accept warnings-checkbox.

The Voters and Votes pages behaves differently: when the error is shown, as soon as you tick the checkbox, the error disappears. When you then change data in the form, the text and checkbox to accept warnings disappears.

The behavior should now be the same across sections, though perhaps slightly different from main.

[oliver3]

I've reached an illegal state with the steps below, which results in a backend error Invalid data error: Data error: recounted==true but voters_recounts is None. This does not occur on main branch.

• start data entry
• enter recounted false
• continue to voters and votes
• enter some value, do not continue
• navigate back to recounted
• enter recounted true
• navigate using the navigation list to voters and votes
• modal "unsaved changes" appears, select save
• error will be shown "Sorry, er ging iets mis, De invoer is niet geldig"
• backend throws an error "Invalid data error: Data error: recounted==true but voters_recounts is None"

[oliver3]

Illegal state recounted==true but voters_recounts is None seems to be fixed!

[jschuurk-kr]

On the Differences page, if the error "Je kan alleen verder als je het het papieren proces-verbaal hebt gecontroleerd." is shown and the accept warnings-checkbox is ticked, changing the data in the form does not untick the accept warnings-checkbox.
The Voters and Votes pages behaves differently: when the error is shown, as soon as you tick the checkbox, the error disappears. When you then change data in the form, the text and checkbox to accept warnings disappears.

The behavior should now be the same across sections, though perhaps slightly different from main.

Created a new issue to add RTL tests for this behavior: #1137

[jschuurk-kr]

@lkleuver Not sure if it's in scope for this PR, but if I have an expired session, I can still navigate through the form. And as part of each new page render, I get the "De sessie is ongeldig" error popup.

I'd expect not to be able to navigate to a different page with an expired session.

[praseodym]

@lkleuver Not sure if it's in scope for this PR, but if I have an expired session, I can still navigate through the form. And as part of each new page render, I get the "De sessie is ongeldig" error popup.

I'd expect not to be able to navigate to a different page with an expired session.

Not in scope for this PR, and also not something that was changed in this PR. When navigating to other parts of the form no requests are made (only when saving, either through the 'Volgende' button or the modal) so no session validation is done.

[praseodym]

It looks like the behaviour of the [X] button in the abort modal has changed compared to main. (Thanks to @jschuurk-kr for finding this one)

Reproduce using:

1. Start data entry
2. Click one of the radio buttons on the "Is er herteld?" page
3. Click "Invoer afbreken", abort modal shows
4. Click [X] to close modal

What happens:

• Modal closes
• Form is saved and next page ("Aantal kiezers en stemmen") is shown

Expected:

• Modal closes, nothing else happens