Generate docs and files for static websites

[gruberb]

The initial idea for this PR comes from wanting better documentation around UniFFI.

Since this repo is handling building the swift components from application-services, this is the right place to facilitate this endeavor. We have loosely coupled .swift files we would like to get documentation out and serve them on a web server.

However, Swift is not allowing us to strip-out the doc comments and serve them on as a static website.

These are the steps involved:

• We have to find the right Package.swift setup so we are able to build a full project. The existing one (from application-services) is not able to do so.
• Generate a .xcframework file from an existing application-services repository.
• Swift is using DocC as their documentation model, which is tightly integrated in XCode. We need to find CLI tools which can take a .xcodeproject file and generate a .doccarchive
• Out of a .doccarchive, which can be browsed through locally, we need to generate a static-website.
• To be able to build application-services, we need to find the right CI/CD environment to build the NSS package + the XCode dependencies
• Once the .xcodeproject file is generated, we need to be able to generate a .doccarchive in the CI/CD workflow and generate static website files from it
• Once this is generated, we push the static website files to a new branch
• We have to sanitize the folder and file names somehow...

All files are under automation/build-docs.

Milestones

• I adapted the Package.swift file to add Glean as a dependency, and updated it to iOSv15. Afterwards, I was able to build a .xcodeproject from a local application-services project.
• XCode comes with CLI tools to build both a .doccarchive and generate a static website from it:

$ xcodebuild docbuild -scheme $SCHEME -destination $DESTINATION -derivedDataPath $OUTPUT_DIR

$ xcrun docc process-archive transform-for-static-hosting $DOCC_ARCHIVE_PATH --output-path $STATIC_FILES_FOLDER --hosting-base-path $HOSTING_PATH

Moving this to the CI/CD platform, there are many, many challenges:

• We need to find the right combination of Python and XCode version so the NSS library builds successfully.
• There are many system dependencies we need to install, which was sometimes interfering with the standard brew installation of Python.
• application-services comes with build scripts, which will install x86_64, and arm64 dependencies. As well as it installs the ios-simulator target on x86_64, which broke the countless builds in many different ways.

The only solution I could see was to "overwrite" the build scripts from application-services and apply them during build time in the GitHub workflow:

cp ./automation/build-docs/overwrites/build-all-ios.sh /tmp/application-services/libs/
cp ./automation/build-docs/overwrites/build-nss-ios.sh /tmp/application-services/libs/
cp ./automation/build-docs/overwrites/verify-ios-ci-environment.sh /tmp/application-services/libs/   
cp ./automation/build-docs/overwrites/build-and-test-swift.py /tmp/application-services/taskcluster/scripts/
cp ./automation/build-docs/overwrites/build-xcframework.sh /tmp/application-services/megazords/ios-rust/

These overwrites remove all ios-simulator and x86_64 targets, and just build for arm64.

We then can go ahead and setup the build environment:

cd /tmp/application-services
./libs/verify-ios-ci-environment.sh

Roadblock

Folder and file names of the generated static website contain not valid characters:

Run actions/upload-artifact@v3
...
Artifact name is valid!
Error: Artifact path is not valid: /data/documentation/mozillaappservices/accesstokeninfo/!=(_:_:).json. Contains the following character:  Colon :
          
Invalid characters include:  Double quote ", Colon :, Less than <, Greater than >, Vertical bar |, Asterisk *, Question mark ?, Carriage return \r, Line feed \n

Many folders and files are named !=(_:_:), and : is not allowed as part of the filename.

[gruberb]

Related issue for swift-docc: swiftlang/swift-docc#284

A possible solution could be to use jazzy for documentation: https://github.com/realm/jazzy

[gruberb]

Closing this in favor of: mozilla/application-services#6354.

It seems much simpler to generate the docs inside application-services. We might be able to replicate the setup here if it's not suitable. But for now, this complex proof-of-concept can be closed for a much leaner version.