HtmlUnit 4.7.0 Failing To Load Forms

[AbdenourAchab]

If you inspect https://www.asr.pima.gov/ using Chrome on Windows 11, you will see that the page has two forms.

But if you run the following code on Apache Netbeans 25 using JDK 24 and HtmlUnit 4.7.0, you will see that no forms are loaded:

package pima;

import org.htmlunit.*;
import org.htmlunit.html.*;

public class Pima {
    public static void main(String[] args) {
        try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX)) {
  java.util.logging.Logger.getLogger("org.htmlunit").setLevel(java.util.logging.Level.OFF);
            webClient.getOptions().setCssEnabled(false);
            webClient.getOptions().setThrowExceptionOnScriptError(false);
            webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
            HtmlPage page = webClient.getPage("https://www.asr.pima.gov/");
            webClient.waitForBackgroundJavaScriptStartingBefore(1_000_000);
            page = (HtmlPage) page.getEnclosingWindow().getEnclosedPage();
            System.out.println("page.asXml() = " + page.asXml());
        }
        catch (Exception e) {
            System.out.println("Exception:  " + e.toString());
        }    
    }    
}

The following does NOT fix the issue:
Adding the following line before "HtmlPage page ...":
webClient.setAjaxController(new NicelyResynchronizingAjaxController());

[rbri]

I'm currently working on deploying the 4.10 release. Will have a look after that.

[AbdenourAchab]

By the way, the reason I am using HtmlUnit 4.7.0 instead of 4.9.0 is that, when I downloaded 4.9.0, I was unable to find the numerous jar files like 4.7.0 had. I found only one jar far, and, when I added that single file to my project library in Netbeans, I was getting errors such as package htmlunit not found. Hopefully, the upcoming 4.10.0 release will have all the jar files I need to upgrade from 4.7.0 to 4.10.0

[rbri]

@AbdenourAchab which file have you downloaded from where,?

[AbdenourAchab]

@rbri , I downloaded htmlunit-4.9.0-bin.zip, htmlunit-4.9.0-src.zip and htmlunit-4.9.0-OSGi.jar from https://sourceforge.net/projects/htmlunit/. I just double checked, and htmlunit-4.9.0-bin\lib does have all the required jar files. For whatever reason, when I copied the extracted htmlunit-4.9.0-bin folder from my Downloads folder to my libraries\external\htmlunit\4.9.0 folder, my Windows only copied the apidocs subfolder and the LICENSE file. It didn't copy the lib subfolder. Maybe I tried to do the copying before the extraction of the zip file was completed.

I have just upgraded my HtmlUnit from 4.7.0 to 4.9.0, and the problem described in the present ticket persists.

[rbri]

Thanks for checking

[AbdenourAchab]

@rbri I have upgraded to HtmlUnit 4.10.0, and the problem described in the present ticket still persists.

[rbri]

Hi @AbdenourAchab,

java.util.logging.Logger.getLogger("org.htmlunit").setLevel(java.util.logging.Level.OFF);

when facing problems, disabling the loggger is the wrong approach.....

Have done some checks and this seems to be a SPA based on angular. This means this uses huge js libraries and browser features not really supported by HtmlUnit so far. We are constantly working on that but at the moment there is no real hope for a solution during the next weeks.

Sorry for that....