Issue 256 - Improve getting hosts performance

[mreynolds389]

There is no need to call "host_show" for each hostname, when "host_find" with the "all" parameter does the same thing.

Signed-off-by: Mark Reynolds mreynolds@redhat.com

fixes: #256

[mreynolds389]

We might be able to apply the same approach for "users", but that needs more investigation...

[pvoborni]

I think this will be faster, but only because the concept is not implemented well in user page.

Combination of sizelimit: 0 with all: true might be potentially very performance intensive as the FreeIPA API might be doing additional operations and LDAP calls for each entry when all: true is provided. Thus I don't think this should be used.

E.g. see:

• https://github.com/freeipa/freeipa/blob/master/ipaserver/plugins/host.py#L1152
• https://github.com/freeipa/freeipa/blob/master/ipaserver/plugins/host.py#L664

The old UI's idea with XXX-find & XXX_show is:

• do XXX_find with pkeys_only: true & size_limit: 0 to get primary keys of all* objects of the type. (the call also returns DN which we don't need). This is only for paging.
• apply paging
• do XXX-show only for the XXX objects to be displayed on the page. It is with or without all: true - that depends if all: true returns some value which is needed to display or for other operations.

User page in new UI has it implemented wrongly as it doesn't do the paging part correctly - instead of calling user_show only for the page, it does it for all and that is the performance problem.

Alternatives I know to this "old UI" behavior are:

• do not do paging at all and use host-find with or without all: true without providing a sizelimit (and thus using the default configured, e.g. 100).
• implemented proper paging on server side and use it

all*: it's actually not all because of default DS sizelimit (2000) for non directory manager queries.

[mreynolds389]

@pvoborni - I agree we need paging (or ideally VLV), but that will probably be a huge change.

For the hosts page, with only 10 hosts, it takes 6-8 seconds to load - that's really slow for only 10 entries! I can't imagine how long it would take for just 100 hosts (or a 1000). With this change it only takes 2 seconds. Even if it's only a temporary solution it's better than what we have now. But I hear what you are saying, and we can close this PR.

[pvoborni]

I agree that doing proper paging is a huge change and thus not on the table atm.

To clarify the current actions: do I understand you correctly that you agree that the thing to do now are:

• not use this MR
• fix the behavior for users to apply the paging properly and use the same solution for hosts and other pages in future?

Btw on the timings. With 11 users in my test setup, I'm getting similar timings in old&new UI (expected as it is still 1 page). And that's about 1.1s - 1.4s total time for both requests - me being in Europe and the server in US (130 ms ping to the server).

[mreynolds389]

Btw on the timings. With 11 users in my test setup, I'm getting similar timings in old&new UI (expected as it is still 1 page). And that's about 1.1s - 1.4s total time for both requests - me being in Europe and the server in US (130 ms ping to the server).

To clarify, I didn't actually test "users", but the "hosts" performance is just awful for so few entries. And this is all on my laptop (no network involved). It's at least a 75% improvement with this PR. Oh well, we'll just have to wait for paged search solution.

[pvoborni]

FYI: Another optimization that the old UI is using, but the new UI is not, is adding option no_members: true for the host_show calls on the "search" pages. This increases the performance as it skips evaluation of associations - that are not needed on those pages.