Merge pull request #214 from mlibrary/docs

Add technical documentation
mlibrary · Sep 27, 2024 · 98c83ef · 98c83ef
2 parents 64b3daf + ec6d89b
commit 98c83ef
Show file tree

Hide file tree

Showing 12 changed files with 463 additions and 538 deletions.
diff --git a/README.md b/README.md
@@ -1,6 +1,63 @@
 # Dromedary -- Middle English Dictionary Application
 
-A new discovery system for the Middle English Dictionary.
+A new(-ish, these days) discovery system for the Middle English Dictionary.
+
+Confusingly, there are three separate repositories:
+ * [dromedary](https://github.com/mlibrary/dromedary), this repo, is the
+   **Rails application**. The name was given the project
+   when someone decided we should start naming project with nonsense words.
+ * [middle_english_dictionary](https://github.com/mlibrary/middle_english_dictionary) is
+   not, as one might expect, the Middle English Dictionary code. Instead,
+   it's the code that pulls out indexable data from each little 
+   XML file, inserts things like links to the OED and DOE, and serves
+   as the basis for building solr documents.
+ * [middle-english-argocd](https://github.com/mlibrary/middle-english-argocd)(_private_) is the argocd setup which deals with environment
+   variables and secrets, and serves to push the application to production. It also
+   has a small-but-valid .zip file under the `sample_data` directory.
+
+## Documentation
+* [Setting up a development environment](docs/setting_up.md) runs through
+  how to get the docker-compose-based dev environment off the ground and
+  index some data into its local solr. 
+* [Overview of indexing](docs/indexing.md) talks about what the indexing
+  process does, where the important files are, and what code might be
+  interesting.
+* [Configuration](docs/configuration.md) does a _very_ brief run through
+  the important ENV values. In general, the [compose.yml](compose.yml) file,
+  the argocd repository, and especially [lib/dromedary/services.rb](lib/dromedary/services.rb)
+  will be the best place to see what values are available to change. _Don't do that
+  unless you know what you're doing, though_.
+* [Solr setup](docs/solr_setup.md) looks at the interesting bits of the
+  solr configuration, in particular the suggesters (for autocomplete).
+* [Tour of the application code](docs/application_code.md) is a quick look at how
+  the MED differs from a stock Rails application.
+* [Deployment to production](docs/deployment.md) shows the steps for building the 
+  correct image and getting it running on the production cluster, as well as
+  how to roll back if something went wrong.
+
+### Access links
+* **Public-facing application**: https://quod.lib.umich.edu/m/middle-english-dictionary/
+* **"Preview" application with exposed Admin panel**: https://preview.med.lib.umich.edu/m/middle-english-dictionary/admin
+
+### About upgrading
+
+This repo currently runs on Ruby 2.x and Blacklight 5.x, and there are no plans
+to upgrade either.
+
+
+<pre>
+
+
+
+</pre>
+
+<hr>
+<hr>
+
+
+
+
+# OLD STUFF 
 
 * [Indexing new data](docs/indexing.md), when new data is made available.
 
@@ -52,85 +109,17 @@ docker-compose build --build-arg ARCH=arm64 solr
 ```shell
 docker-compose up -d
 ```
-NOTES
-* The ***sidekiq*** container will exit because we have yet to do a ***bundle install!***.
-> ### Install bundler
-> ```shell
-> docker-compose exec -- gem install 'bundler:~>2.2.21'
-> RUN bundle config --local build.sassc --disable-march-tune-native
-> ```
-> This was moved into the Dockerfile so it is no longer is necessary but is left here as a reminder so it will not be forgotten.
->
-> Need to revisit why setting the bundler version is necessary in the first place!
-### Configure bundler
-```shell
-docker-compose exec -- app bundle config set rubygems.pkg.github.com <personal-access-token>
-```
-The above command creates the following file: ./bundle/config
-```yaml
----
-BUNDLE_RUBYGEMS__PKG__GITHUB__COM: "<personal-access-token>"
-```
-NOTES
-* [Personal access tokens (classic)](https://github.com/settings/tokens) Token you have generated that can be used to access the [GitHub API](https://docs.github.com/en) -- read:packages.
-* Replace <personal-access-token> with your personal access token.
 
-### Bundle install
-```shell
-docker-compose exec -- app bundle install
-```
-NOTES
-* Environment variable **BUNDLE_PATH** is set to **/var/opt/app/gems** in the **Dockerfile**.
-### Yarn install
-```shell
-docker-compose exec -- app yarn install
-```
-NOTES
-* Investigate using a volume for **node_modules** directory like we do for **gems**
-### Setup databases
-```shell
-docker-compose exec -- app bundle exec rails db:setup
-```
-If you need to recreate the databases run db:drop and then db:setup.
-```shell
-docker-compose exec -- app bundle exec rails db:drop
-docker-compose exec -- app bundle exec rails db:setup
-```
-NOTES
-* Names of the databases are defined in **./config/database.yml**
-* The environment variable **DATABASE_URL** takes precedence over configured values.
-### Create solr collections
-```shell
-docker-compose exec -- solr solr create_collection -d dromedary -c dromedary-development 
-docker-compose exec -- solr solr create_collection -d dromedary -c dromedary-test 
-```
-If you need to recreate a core run delete and create_core (e.g. dromedary-test)
-```shell
-docker-compose exec -- solr solr delete -c dromedary-test
-docker-compose exec -- solr solr create_collection -d dromedary -c dromedary-test 
-```
-NOTES
-* Names of the solr cores are defined in **./config/blacklight.yml** file.
-* The environment variable **SOLR_URL** takes precedence over configured values.
-### Start development rails server
-```shell
-docker-compose exec -- app bundle exec rails s -b 0.0.0.0
-```
+
 Verify the application is running http://localhost:3000/
+
 ## Bring it all down then back up again
 ```shell
 docker-compose down
 ```
 ```shell
 docker-compose up -d
 ```
-```shell
-docker-compose exec -- app bundle exec rails s -b 0.0.0.0
-```
-The gems, database, and solr redis use volumes to persit between the ups and downs of development.
-When things get flakey you have the option to simply delete any or all volumes after you bring it all down.
-If you remove all volumes just repeat the [Development quick start](#development-quick-start), otherwise
-you'll need to run the appropriate steps depending on which volumes you deleted:
-* For gems run the [Bundle install](#bundle-install) step.
-* For database run the [Setup databases](#setup-databases) step.
-* For solr run the [Create solr collections](#create-solr-collections) step.
+
+Note that there's no data in it yet, so lots of actions will throw errors. It's time
+to index some data.
diff --git a/app/controllers/catalog_controller.rb b/app/controllers/catalog_controller.rb
@@ -187,7 +187,12 @@ class CatalogController < ApplicationController
     # solr request handler? The one set in config[:default_solr_parameters][:qt],
     # since we aren't specifying it otherwise.
 
-    # config.add_search_field 'Keywords', label: 'Everything'
+
+    ######################### WHAT ARE THE DOLLAR-SIGN VARIABLES??? ############
+    # These are sent to solr as the actual string (e.g., solr gets "$everything_qf").
+    # They are then expanded by solr withing the solr process based on configuration
+    # files there. See, e.g., solr/dromedary/conf/solrconfig_med/entry_searches/everything_search.xml
+    # for an example.
 
     config.add_search_field("anywhere", label: "Entire entry") do |field|
       field.qt = "/search"

diff --git a/docs/application_code.md b/docs/application_code.md
@@ -0,0 +1,52 @@
+# Tour of the application code
+
+In general, the MED is a "normal" Blacklight (v5) application, with a lot
+of added stuff.
+
+Like most Blacklight apps, the heart of the configuration is
+in the [Catalog Controller](../app/controllers/catalog_controller.rb), 
+specifying the search and facet fields. This is repeated for the
+other controllers, and they're fairly straightforward so long as
+you're willing to take on faith that Rails will find things at the
+right time.
+
+
+## Models
+The files in [models](../app/models) are unusual for a Rails app in that we're not using
+a backing database, and thus not deriving them from ActiveRecord.
+The directly is empty of anything interesting.
+
+The _actual_ objects to represent each of the many, many layers of a dictionary 
+entry are actually defined in the [middle_english_dictionary](https://github.com/mlibrary/middle_english_dictionary)
+repository. The nomenclature can be confusing, since it's derived from 
+the jargon of the field, but none of the objects are particularly complex.
+
+## Presenters
+
+The meat of the interface is actually built into the presenters.
+[quotes presenter](../app/presenters/quotes/index_presenter.rb) 
+is indicative of the setup, pulling in lots of XSLT and getting values
+out of the document with XSLT transforms or by querying the 
+`Nokogiri::XML` document directly. 
+
+## lib/med_installer
+
+...is all indexing code, and isn't used in the rest of the application. See
+[indexing.md](indexing.md) for a brief overview.
+
+## /solr
+
+...has all the solr configuration in it, as well as the
+[Dockerfile](../solr/Dockerfile) and the [container initialization](../solr/container/solr_init.sh)
+code. See [solr.md](solr.md) for more.
+
+## Everything else
+
+...is basically just normal views and utility objects
+(most importantly the [services.rb](../lib/dromedary/services.rb) file). 
+
+
+
+
+
+
diff --git a/docs/autocomplete_setup.md b/docs/autocomplete_setup.md
@@ -4,90 +4,58 @@ The blacklight autocomplete setup is pretty brittle and needs some mucking
 about with to get it to work with multiple different input boxes that
 target different solr autocomplete endpoints. 
 
-## Three things that need doing
+## Three things that needed doing
 * Add a new autocomplete handler in the solr config
 * Make changes to `config/autocomplete.yml`
-* Add the option to the dropdown where the users determines what to search
-
-If you're adding a whole new autocomplete input field in the HTML,
-you'll also need to:
-* Create the input box with  
-* Make changes to `autocomplete.js.erb` 
-
+* Add javascript code to trigger autocomplete
+  to the dropdown where the users determine what to search
 
 ## Configure the new autocomplete handler in the solr config
 
-Suggesters live in `solr/med/conf/solrconfig_med/suggesters`. You 
-can pattern a new one off of the ones in there.
+Suggesters live in 
+[solr/dromedary/conf/solrconfig_med/suggesters](../solr/dromedary/conf/solrconfig_med/suggesters). 
+You can pattern a new one off of the ones in there.
 
 Of course, if you're using an existing handler in a new context (say,
 you want autocomplete for headwords again in an advanced search box), you 
 can just use the handler that's already been defined and skip ahead to 
 making and configuring the new search input field and dropdown.
 
 Things to note:
-* The name in `<str name=-name...>` in the top section is any name you
-make up to identify this suggester. 
-* ... but the `suggest.dictionary` in the bottom section *must* match that name
-* ... and same with the `<arr name="components"...>` in the bottom
+
+* You can pick any names for the suggester handler, but it
+  must be used _identically_ in three places:
+  * The top `<str name=-name...>`
+  * The name of the `suggest.dictionary`
+  * The reference in the `<arr name="components"...>` block.
 * The `field` is the name of the field you're basing this on. It *must*
-be a stored field!
+be a stored field! The code we have now builds a special field for this
+instead of trying to force an existing field to work.
 * `"suggestAnalyzerFieldType"` is probably the same fieldType used for the
 field you're indexing in this typeahead field, but if you have the know-how
 and think it should be different, go for it.
 
 
-## Add new suggester to autocomplete in `config/autocomplete.yml`
-
-```yaml
-# Autocomplete setup.
-# The format is:
-#   search_name:
-#     solr_endpoint: path_to_solr_handler
-#     search_component_name: mySuggester
-#
-# The search_name is the name given the search in the
-# `config.add_search_field(name, ...)` in catalog_controller
-#
-# The "keyword" config mirrors the default blacklight setup
-
-default: &default
-  keyword: 
-    solr_endpoint: path_to_solr_handler,
-    search_component_name: "mySuggester"
-  h:
-    solr_endpoint:         headword_only_suggester
-    search_component_name: headword_only_suggester
-  hnf:
-    solr_endpoint:         headword_and_forms_suggester
-    search_component_name: headword_and_forms_suggester
-  oed:
-    solr_endpoint:         oed_suggester
-    search_component_name: oed_suggester
-
-development:
-  <<: *default
-
-test:
-  <<: *default
-
-production:
-  <<: *default
-
-```
+## Add new suggester to the autocomplete configuration
+Pattern match from another entry 
+and add it to [autocomplete.yml](../config/autocomplete.yml)
 
 ## Catalog controller configuration
 
-Now load the autocomplete setup into your blacklight configuration.
-
 ```ruby
 
 # Autocomplete on multiple fields. See config/autocomplete.yml
 config.autocomplete = ActiveSupport::HashWithIndifferentAccess.new Rails.application.config_for(:autocomplete)
 
 ```
 
-## Adding a whole new dropdown
+## Adding a whole different search box
+
+I don't expect this will happen at this point, but the knowledge may 
+well come in handy on other project.
+
+### Adding a whole new dropdown
+
 * Put a data attribute `data-autocomplete-config` on your text box
     to reference which typeahead configuration should be used (e.g,
     `h` or `hnf` in the config example above). 
@@ -116,7 +84,5 @@ the correct index when a user picks, e.g., "headword only."
 ## Side note: this overrides blacklight code
 
 I don't actually moneky-patch, but I do use `Module#prepend`. The code
-is in `config/initializers/autcomplete_override_code.rb`
+is in [autocomplete_override_code.rb](../config/initializers/autocomplete_override_code.rb).
 
-If Blacklight ever changes the autocomplete setup to allow this sort of
-thing, we'll need to re-evaluate whether these extensions are necessary.