analytics

docs/analytics/00-intro.md

@@ -46,18 +46,47 @@ Detects if an entity is added, modified, or deleted. It can then automatically
 update the summary table accordingly. No need to devise a way to signal an
 external system that the data has changed. Or even to blindly recalculate all
 data up to 5 years ago periodically, just because you fear some process might
-have changed records that old, but you can't know for sure.
+have changed records that old, but you can't know for certain.
 
 ## Requirements
 
-Only works with PostgreSQL for now.
+Only works with PostgreSQL for now. Only the usage under Symfony is supported
+for now.
 
 ## Installation
 
+Install the bundle using Composer:
+
 ```bash
 composer require rekalogika/analytics-bundle
 ```
 
+Add the necessary DQL functions:
+
+```yaml title="config/packages/doctrine.yaml"
+doctrine:
+    orm:
+        dql:
+            string_functions:
+                REKALOGIKA_NEXTVAL: Rekalogika\Analytics\Doctrine\Function\NextValFunction
+                REKALOGIKA_TRUNCATE_BIGINT: Rekalogika\Analytics\Doctrine\Function\TruncateBigIntFunction
+                REKALOGIKA_GROUPING_CONCAT: Rekalogika\Analytics\Doctrine\Function\GroupingConcatFunction
+                REKALOGIKA_NULL: Rekalogika\Analytics\Doctrine\Function\NullFunction
+                REKALOGIKA_CAST: Rekalogika\Analytics\Doctrine\Function\CastFunction
+            numeric_functions:
+                REKALOGIKA_DATETIME_TO_SUMMARY_INTEGER: Rekalogika\Analytics\Doctrine\Function\DateTimeToIntegerFunction
+                REKALOGIKA_TRUNCATE_UUID_TO_BIGINT: Rekalogika\Analytics\Doctrine\Function\TruncateUuidToBigintFunction
+```
+
+Configure Symfony Messenger routing configuration. For example:
+
+```yaml title="config/packages/messenger.yaml"
+framework:
+    messenger:
+        routing:
+            Rekalogika\Analytics\RefreshWorker\RefreshCommand: async
+```
+
 ## License
 
 MIT

docs/analytics/99-internals/02-refresh.md

@@ -1,21 +1,30 @@
 ---
-title: Refresh
+title: Refresh Mechanism
 ---
 
-## `SummarySignal`
+## `DirtyFlag`
 
-A `SummarySignal` is an entity indicating a dirty partition, which is a
-partition in a summary table that needs to be refreshed. `SummarySignal` should
-be persisted and flushed at the same time as the rest of the entities, during
-the `onFlush` event. So if the flush fails, the signal will not be saved.
+A `DirtyFlag` is an entity indicating a dirty partition, which is a partition in
+a summary table that needs to be refreshed. `DirtyFlag` should be persisted and
+flushed at the same time as the rest of the entities, during the `onFlush`
+event. So if the flush fails, the flag will not be saved.
 
-A `SummarySignal` can have null partition information. In this case, it means
-there are new persisted entities that need to be summarized. New entities
-need to be handled differently because they are not guaranteed to have an
-ID before flush.
+A `DirtyFlag` can have null partition information. In this case, it means there
+are new persisted entities that need to be summarized. New entities need to be
+handled differently because they are not guaranteed to have an ID before flush.
 
 ## `SourceEntityListener`
 
 Listens on Doctrine's `onFlush` event. For every pending entities, it checks the
-change set and determines if the change affects any summary tables. Then it
-instantiates the corresponding `SummarySignal` objects and persists them.
+change set and determines if the change affects any summary tables. If a
+property is changed, and that would affect one or more summary tables, it
+creates the corresponding `DirtyFlag` objects for each of the partition that
+needs to be refreshed.
+
+:::warning
+
+This automatic detection works only for simple cases: it only detects the change
+of the properties in the entity itself. If the change is in a related entity, it
+will not
+
+:::

docs/analytics/99-internals/03-todo.md

@@ -0,0 +1,33 @@
+---
+title: Current To-do List
+---
+
+## Bugs
+
+* Max and min values do not consider `HasQueryBuilderModifier`
+* Canonicalize grouping column ordering (use alphabetical order?), avoid relying
+  on undefined reflection ordering.
+
+## Missing Features
+
+* Doctrine ORM 3 support
+* Metadata caching
+* Disable refresh event dispatching in manual refresh
+* Support for MS SQL Server
+* Configurable event timing (`RefreshClassPropertiesResolver`)
+* Allow customized generation of dirty flag (see `DirtyFlagGenerator`)
+* It should be possible for `DirtyFlagGenerator` to generate dirty partition
+  signals for non-`IDENTITY` entities, bypassing 'new entity' phase.
+* Hyperloglog
+* Binning value resolver
+* Measure using getter (example: getAverage() that gets its value from `$this->sum / $this->count`)
+
+## Limitations
+
+* Postgres `GROUP BY` maximum grouping set limit of 4096, can be circumvented by
+  using multiple queries.
+
+## Reorganization
+
+* Spin off pivot table to its own package.
+* Exception refactoring