scheduled stop

.gitignore

@@ -1,7 +1,7 @@
 *.o
 .pytest_cache
 Makefile.local
-apsis.db
+apsis.db*
 apsis.log
 archive*.db
 *-journal

docs/concepts.rst

@@ -99,6 +99,7 @@ Each run, once created, is in one of these states:
 - **waiting**: The run is waiting for a condition to be met.
 - **starting**: The run is starting.
 - **running**: The run has started and is currently running.
+- **stopping**: Apsis is stopping the run.
 - **success**: The run has completed successfully.
 - **failure**: The run has completed unsuccesfully.
 - **error**: Some other problem has occured with the run.  This can include a
@@ -149,6 +150,16 @@ You can apply the following operations, to induce transitions explicitly:
 - You can *skip* a **scheduled** or **waiting** run.  Apsis no longer waits for
   its schedule time or conditions, and transitions it to **skipped**.
 
+- You can *stop* a **running** run.  Apsis requests that the run shut down in an
+  orderly manner.  How this works depends on the run's program.  For a program
+  that runs a (local or remote) UNIX process, this entails sending a termination
+  signal (usuall SIGTERM), then waiting for a grace period and then sending
+  SIGKILL if the process has not terminated.  While Apsis is waiting for the run
+  to terminate, it is in the **stopping** state.
+
+  You can also schedule Apsis to stop a run automatically; see
+  :ref:`stop-schedules`.
+
 - You can *mark* a finished run (**success**, **failure**, **skipped**, or
   **error**) to a different finished state.
 

docs/programs.rst

@@ -163,6 +163,53 @@ the user running the Procstar agent to run the command as the sudo user, without
 any explicit password.
 
 
+.. _program-stop:
+
+Program Stop
+------------
+
+Many program types provide a stop method, by which Apsis can request an orderly
+shutdown of the program before it terminates on its own.  Keep in mind,
+
+- Not all program types provide a program stop.
+- The program may not stop immediately.
+- The program stop may fail.
+
+Apsis requests a program to stop if the program's run is configured with a stop
+schedule, or in response to an explicit stop operation invoked by the user.
+
+Before Apsis requests a program to stop, it transitions the run to the
+*stopping* state.  If the program terminates correctly in response to the stop
+request, Apsis transitions the run to *success*; if the program terminates in an
+unexpected way, *failure*.
+
+The program types above that create a UNIX process (`program`, `shell`,
+`procstar`, `procstar-shell`) all implement program stop similarly.  In response
+to a program stop request,
+
+1. Apsis immediately sends the process a signal, by default `SIGTERM`.
+2. Apsis waits for the process to terminate, up to a configured grace period, by
+   default 60 seconds.
+3. If the process has not terminated, Apsis sends it `SIGKILL`.
+
+To configure the program stop, use the `stop` key.  For example, Apsis will
+request this program to stop by sending `SIGUSR2` instead of `SIGTERM`, and will
+only wait 15 seconds before sending `SIGKILL`.
+
+.. code:: yaml
+
+    program:
+        type: procstar
+        group_id: default
+        argv: ["/usr/bin/echo", "Hello, world!"]
+        stop:
+            signal: SIGUSR2
+            grace_period: 15s
+
+If the program terminates with an exit status that indicates the process ended
+from `SIGUSR2`, Apsis considers the run to have succeeded.
+
+
 Internal Programs
 -----------------
 

docs/schedules.rst

@@ -210,3 +210,54 @@ transition, the schedule will include no times on this date at all.  For
 example, a daily schedule with a start time between 2:00:00 and 3:00:00 and a
 U.S. time zone will contain no times on the dates in the spring when DST begins.
 
+
+.. _stop-schedules:
+
+Stop schedules
+--------------
+
+You can also configure a job so that a run will stop at a certain time.  When
+this time elapsis, Apsis stops the run, assuming it is running.  The ways you
+schedule a run to stop are different from the schedule types above, since the
+stop time is generally related to the schedule time.
+
+To configure a stop schedule in addition to the normal start schedule, place the
+latter into `start:` subkey, and use `stop:` to specify the stop schedule.  The
+available stop schedule types are `duration` and `daytime`.  For example,
+
+.. code:: yaml
+
+    schedule:
+        start:
+            type: daily
+            daytime: 10:30:00
+            tz: Europe/Berlin
+        stop:
+            type: duration
+            duration: 30m
+
+This instructs Apsis to run the program for 30 minutes after the schedule time,
+namely 11:00 Europe/Berlin.
+
+The `daytime` stop schedule type instructs Apsis to stop the run at the next
+occurence of a specific daytime after the schedule time.  The following example
+has the same effect as the previous:
+
+.. code:: yaml
+
+    schedule:
+        start:
+            type: daily
+            daytime: 10:30:00
+            tz: Europe/Berlin
+        stop:
+            type: daytime
+            daytime: 11:00:00
+            tz: Europe/Berlin
+
+You can use any of the schedule types in the previous section for the `start`
+schedule.
+
+When the stop time elapses, Apsis stops the run in accordance with the program's
+stop method.  This is discussed in :ref:`program-stop`.
+

notes/notes.md

@@ -1,3 +1,34 @@
+# Scheduled stop
+
+THIS IS CORRECT:
+```yaml
+program:
+  type: agent
+  argv: ["/path/to/my/service", "--foreground"]
+  stop:
+    signal: SIGTERM
+    grace_period: 1m
+    self_stop: false
+
+schedule:
+  start:
+    type: interval
+    interval: 1h
+  stop:
+    type: duration
+    duration: 30m
+```
+
+
+|           | running | stopping |
+|-----------|---------|----------|
+| exit ==0  | success | success  |
+| exit !=0  | failure | failure  |
+| stop sig  | failure | success  |
+| other sig | failure | failure  |
+
+
+
 # SQLite
 
 Performance test of appending to string fields, in `work/sqlite-concat.py`.

notes/todo.md

@@ -1,24 +1,3 @@
-# Live updates
-
-WebSocket API for web UI:
-- run updates for all runs (as current)
-- run log updates for a single run
-- run data updates for a single run
-- agent changes: connect, disconnect, timeout
-- job changes
-
-Persistent connection:
-- run changes (run summaries only)
-- job changes
-- Procstar connection changes
-- Apsis log (if we keep this)
-
-Transient connection (single run):
-- run log updates
-- run output data updates
-- run metadata updates
-
-
 ### The Plan
 
 - [x] design internal run publisher protocol
@@ -54,26 +33,29 @@ Transient connection (single run):
   - [x] move compression out of programs
 - [x] roll in Procstar agent changes to `/summary`
 - [ ] clean up API endpoints we don't need anymore
-- [ ] roll in, or get rid of, log socket
+- [x] roll in, or get rid of, log socket
 - [x] add live endpoints to `Client`
 - [x] live updates in CLUI
   - [x] output
   - [x] run log
 - [x] improve and clean up `State` enum
 
+- [x] scheduled stop
+  - [x] add stop method to program
+  - [x] actually stop the program at the stop time
+  - [x] go through _stopping_ state
+  - [x] Procstar stop method
+  - [x] add stop time to runs table
+  - [x] classic agent program stop method
+  - [x] ProcessProgram: RunningProgram and stop method
+  - [x] stop run API endpoint
+  - [x] "stop" operation
+  - [x] add stop time to run view
+  - [x] docs
+  - [x] refactor `apsis.stop` module
 - [ ] if `send_signal` raises, error the run
-- [ ] improve `apsis job` output style
-
 
-# Exantium tasks?
-
-- internals presentation
-- reliable integration tests
-- update Python deps, especially Sanic
-- update JS deps, especially Vue3
-- database schema cleanup
-  - move metadata into its own table; remove `meta` from `Apsis._transition()`
-- code cleanup, reorg, documentation
+- [ ] improve `apsis job` output style
 
 
 # Cleanup