Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve LDAP performances when updating records, especially when many user entries exists #1975

Open
wants to merge 19 commits into
base: dev
Choose a base branch
from

Conversation

zamentur
Copy link
Member

@zamentur zamentur commented Oct 21, 2024

The problem

With 150 users and 12 apps, a yunohost user create take 1min
With 350 users and ~25 apps, a yunohost user create take hours maybe days.

Solution

Replace ldap.modlist.modifyModlist by an optimize code that work for yunohost.

Test in a dev platforme (with ssd 600Mbps) : the 169th user creation is made in ~3s.

PR Status

Need more test to be sure it doesn't break something...

How to test

for i in {1..150}; do echo $i; yunohost user test$i -d ynh.local -F "test test" -p "Thisisadummypa:ssw0rd"; done

@alexAubin alexAubin changed the title Enh perf ldap update Improve LDAP performances when updating records, especially when many user entries exists Oct 21, 2024
src/utils/ldap.py Show resolved Hide resolved
src/utils/ldap.py Outdated Show resolved Hide resolved
@Salamandar
Copy link
Contributor

Not sure about your last commits on [user] -> user, but apart from that, LGTM.

@zamentur zamentur changed the base branch from dev to main January 20, 2025 14:03
@zamentur zamentur changed the base branch from main to dev January 20, 2025 14:04
conf/slapd/permission.ldif Show resolved Hide resolved
src/settings.py Show resolved Hide resolved
Comment on lines 488 to +491
# If the requested mail address is already as main address or as an alias by this user
if mail in user["mail"]:
user["mail"].remove(mail)
if mail != user["mail"][0]:
user["mail"].remove(mail)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure to fully understand that bit but my brain is not super fresh right now

Comment on lines 516 to +517
if mail in user["mail"]:
user["mail"].remove(mail)
continue
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also not sure to understand this bit

src/user.py Show resolved Hide resolved
src/user.py Outdated Show resolved Hide resolved
@zamentur zamentur force-pushed the enh-perf-ldap-update branch from 1e1e80f to c7e99af Compare January 28, 2025 11:16
@zamentur
Copy link
Member Author

Tests are ok on my local setup, but on the CI systemctl is-active yunohost-portal-api >/dev/null return 768 instead of 0 code...

@alexAubin @Axolotle : Is it possible that SSE and work on moulinette could have triggered this error ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants