meta data version of the code where documents are chunked and tagged by year and page number

[anujcb]

@mayooear
do you mind sharing the meta data version of the code? where you chunk and tag the documents by year and page number. I am really interested in what you showed in the demo. Impressive work!!!

[anujcb]

const groupDocumentsByYear = (documents) => {
const groups = new Map();
for (const doc of documents) {
try {
console.log('Processing document:', doc.metadata.source);
//const year = doc.filepath.split('/')[1];
console.log('Processing doc before year:', doc.metadata.source.replace(///g, '\'));
const year = doc.metadata.source.replace(///g, '\').split('\')[9];
console.log('Processing doc year:', year);
const docs = groups.get(year) || [];
docs.push(doc);
groups.set(year, docs);
} catch (error) {
console.log(Error occurred while grouping ${doc.filepath}: ${error});
}
}
return groups;
};

[report2model]

Sorry for the basic question, but would this be inserted between the PDF Loader and Split text into chunks constants? Would it require an additional import? Not a programmer so still learning.