Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
source-mongodb: select projections of change stream documents
Rather than including all available fields in change stream documents, apply filtering based on projections to get only the fields we need. The prevents issues such as the `updateDescription` of a document re-stating a huge field that is also in the `fullDocument` and causing errors because the change stream document is over the 16MB limit with the doubly enumerated field. A slightly more thorough solution would also involve the `$changeStreamSplitLargeEvent` pipeline filter, although that option has some limitations: It is only supported on relatively new versions of MongoDB, and it is not completely bulletproof in that it does not traverse and split nested fields. So we would almost certainly want to use these projections to select only the necessary fields even if we used the `$changeStreamSplitLargeEvent` filter too.
- Loading branch information